-- Logs begin at Tue 2022-07-05 14:28:08 UTC, end at Tue 2022-07-05 15:34:59 UTC. -- Jul 05 14:59:14.795698 dv-cinder-841125-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 05 14:59:14.804663 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 05 14:59:14.806639 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 5 14:59:14 2022] *** Jul 05 14:59:14.806639 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 05 14:59:14.806639 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 05 14:59:14.806639 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: nodename: dv-cinder-841125-10-iscsi Jul 05 14:59:14.806639 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: machine: x86_64 Jul 05 14:59:14.806639 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: clock source: unix Jul 05 14:59:14.806639 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: pcre jit disabled Jul 05 14:59:14.806639 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: detected number of CPU cores: 4 Jul 05 14:59:14.806639 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: current working directory: / Jul 05 14:59:14.806639 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: detected binary path: /usr/bin/uwsgi-core Jul 05 14:59:14.807214 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: your processes number limit is 47680 Jul 05 14:59:14.807214 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: your memory page size is 4096 bytes Jul 05 14:59:14.807214 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: detected max file descriptor number: 2048 Jul 05 14:59:14.807214 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: lock engine: pthread robust mutexes Jul 05 14:59:14.807214 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: thunder lock: enabled Jul 05 14:59:14.807214 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 05 14:59:14.807214 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 05 14:59:14.850246 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: Python main interpreter initialized at 0x5583280809c0 Jul 05 14:59:14.850246 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: python threads support enabled Jul 05 14:59:14.850246 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: your server socket listen backlog is limited to 100 connections Jul 05 14:59:14.850246 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: your mercy for graceful operations on workers is 90 seconds Jul 05 14:59:14.850564 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: mapped 403077 bytes (393 KB) for 2 cores Jul 05 14:59:14.850647 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: *** Operational MODE: preforking *** Jul 05 14:59:14.850916 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: *** uWSGI is running in multiple interpreter mode *** Jul 05 14:59:14.850916 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: spawned uWSGI master process (pid: 80345) Jul 05 14:59:14.851116 dv-cinder-841125-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 05 14:59:14.851649 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: spawned uWSGI worker 1 (pid: 80351, cores: 1) Jul 05 14:59:14.851978 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: spawned uWSGI worker 2 (pid: 80352, cores: 1) Jul 05 14:59:14.853889 dv-cinder-841125-10-iscsi devstack@keystone.service[80345]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 05 14:59:20.234995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 14:59:20.234995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: from cryptography.utils import int_from_bytes Jul 05 14:59:20.236269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 14:59:20.236269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: from cryptography.utils import int_from_bytes Jul 05 14:59:20.236750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 14:59:20.236750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: from cryptography.utils import int_from_bytes Jul 05 14:59:20.236918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 05 14:59:20.236918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: from cryptography.utils import int_from_bytes Jul 05 14:59:20.284822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 05 14:59:20.284971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 05 14:59:20.285059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 05 14:59:20.285150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 05 14:59:20.285231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] command line args: [] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 05 14:59:20.285315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] command line args: [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 05 14:59:20.285470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 05 14:59:20.285600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 05 14:59:20.285682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ================================================================================ {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 05 14:59:20.285756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ================================================================================ {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 05 14:59:20.285897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] admin_token = **** {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.285990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] admin_token = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.286146 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] config_dir = [] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.286224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] config_dir = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.286336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.286421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.286524 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] config_source = [] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.286603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] config_source = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.286707 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] debug = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.286784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] debug = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.286905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.287187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.287284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.287345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.287470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.287547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.287706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.287790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.287952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288031 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] list_limit = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] log_config_append = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_config_append = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288718 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] log_dir = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_dir = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_file = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.288970 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] log_file = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.289109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.289194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.289348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.289562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.289669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.289743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.289829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.289956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.290043 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.290131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.290208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.290288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.290399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.290495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.290644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.290742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.290875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.290960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.291128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.291215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.291364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.291463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.291596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.291679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.291843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.291920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.292091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.292181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.292367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.292449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.292854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.292936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.293059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] publish_errors = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.293134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] publish_errors = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.293247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.293343 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.293468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.293568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.293642 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.293718 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.293879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.293987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.294112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.294192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.294344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] standard_threads = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.294481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] standard_threads = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.294557 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.294627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.294798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.294877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.294973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] use_journal = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] use_journal = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] use_json = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] use_json = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] use_stderr = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] use_stderr = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] use_syslog = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] use_syslog = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.295975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 05 14:59:20.296154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.296255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.296391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.296474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.296656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.296740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.296896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.296986 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.297145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.297231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.297380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.297457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.297624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.298224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.298397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.298476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.298646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.298724 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.298891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.298981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.299163 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.299254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.299398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.299483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.299643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.299734 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.299880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.299974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.300125 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.300201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.300370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.300526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.300601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.300678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.300848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.300926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.301098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.301186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.301340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.301417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.301575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.301659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.301827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.301912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.302183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.302259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.302439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.302517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.302689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.302779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.302921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.302999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.303174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.303268 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.303408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.303493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.303643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.303720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.303893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.303976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.304138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.304217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.304367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.304443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.304595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.304671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.304840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.304919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.305086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.305163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.305322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.305404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.305567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.305646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.305797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.305880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.306031 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.306108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.306261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.306335 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.306483 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.306568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.306711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.306787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.306941 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.307033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.307194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.307273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.307431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.307508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.307678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.307764 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.307909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.307987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.308146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.308256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.308363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.308458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.308601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.308711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.308834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.308937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.309082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.309203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.309311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.309388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.309548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.309661 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.309782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.309903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.310016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.310094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.310251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.310328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.310486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.310568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.310715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.310792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.310963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.311097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.311221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.311358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.311439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.311520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.311676 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.311765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.311935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.312013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.312169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.312252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.312398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.312479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.312633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.312720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.312866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.312948 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.313129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.313217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.313910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.314012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.314169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.314249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.314417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.314495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.314651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.314734 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.314884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.314962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.315142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.315231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.315392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.315485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.315626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.315710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.315859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.315937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.316099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.316185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.316333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.316410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.316566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.316652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.316798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.316888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.317032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.317109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.317262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.317344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.317522 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.317608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.317759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.317836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.317996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.318074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.318231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.318384 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.318460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.318532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.318694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.318774 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.318912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.319059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.319175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.319269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.319413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.319493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.319645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.319723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.319875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.319960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.320107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.320189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.320335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.320414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.320574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.320654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.320817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.320896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.321064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.321142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.321298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.321378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.321553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.321630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.321811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.321889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.322050 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.322129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.322281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] auth.external = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.322361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.external = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.322511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.322597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.322806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.322914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.323037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.323132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.323290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.password = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.323372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] auth.password = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.323517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] auth.token = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.323595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] auth.token = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.323754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.323835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.323992 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.324072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.324236 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.324317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.324466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.324546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.324711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.324797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.324955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.325034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.325183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.325264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.325417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.325495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] credential.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.325663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.325742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.325909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.325989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.326153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.326233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.326387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.326465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.326626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.326706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.326859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.326940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.327145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.327230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.327399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.327488 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.327635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.327714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.327886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.328222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.328430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.328510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.328675 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.328754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.328967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.329046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.329214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.329296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.329469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.329548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.329704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.329782 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] federation.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.329938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.330024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.330182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.330261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.330426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.330505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.330655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.330734 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.330898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.330977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.331154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.331235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.331413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.331493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.331647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.331732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.331900 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.331990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.332135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.332214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.332380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.332458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.332614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.332764 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.332860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.333002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.333121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.333200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.333317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.333406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.333550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.333628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.333795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.333878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.334026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.334118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.334263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.334342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.334511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.334589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.334758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.334844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.334988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.335122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.335242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.335322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.335473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.335564 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.335713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.335792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.335960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.336038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.336203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.336290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.336456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.336535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.336700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.336778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.336965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.337053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.337202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.337287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.337435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.337514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.337666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.337754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.337902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.337981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.338134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.338213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.338368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.338456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.338608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.338687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.338845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.338924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.339111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.339206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.339344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.339422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.339589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.339668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.339833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.339922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.340066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.340145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.340310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.340388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.340553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.340641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.340780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.340859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.341015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.341093 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.341242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.341328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.341479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.341558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.341711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.341789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.341949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.342036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.342183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.342261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.342415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.342493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.342660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.342748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.342904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.342982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.343156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.343237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.343385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.343476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.343630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.343709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.343874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.343952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.344106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.344187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.344338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.344417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.344594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.344674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.344823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.344903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.345060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.345145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.345316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.345395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.345545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.345625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.345790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.345868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.346033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.346286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.346376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.346521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.346601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.346752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.346832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.346987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.347091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.347243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.347323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.347477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.347556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.347705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.347792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.347932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.348011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.348177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.348256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.348418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.348505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.348661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.348739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.348904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.348983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.349134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.349222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.349362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.349440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.349605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.349691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.349840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.349927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.350073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.350156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.350307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.350385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.350540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.350629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.350786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.350865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.351024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.351137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.351285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.351376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.351608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.351761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.351839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.351998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.352085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.352231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.352309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.352465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.352542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.352698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.352786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.352936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.353015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.353168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.353248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.353396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.353483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.353632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.353712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.353872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] resource.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.353952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.354118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.354206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.354354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.354434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.354586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.354664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.354834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.354922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.355091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.355173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.355331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.355410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.355565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.355776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.355861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.355934 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.356028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] role.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.356107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.356262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] role.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.356352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] role.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.356493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.356574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] role.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.356728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.356807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.356974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.357061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.357219 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.357299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.357462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.357542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.357709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.357792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.357954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.358034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.358200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.358279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.358444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.358525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.358675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.358755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.358907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.358985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.359177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.359259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.359423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.359503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.359672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.359752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.359926 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.360069 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.360818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.360899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.361073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.361153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.361323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.361407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.361571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.361653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.361817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.361896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.362051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.362132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.362289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.362368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.362543 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.362624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.362779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.362859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.363011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.363111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.363268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.363361 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.363500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.363580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.363734 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.363814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.363986 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.364078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.364231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.364310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.364469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.364549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.364706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.364794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.364937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.365018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.365165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.365243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.365401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.365488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] token.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.365635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.365714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.365869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.365948 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.366107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.366194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.366358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.366437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.366594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.366674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.366830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.366917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.367090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.367174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.367336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.367414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.367569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.367664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.367806 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.367887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.368041 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.368118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.368276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.368364 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.368520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.368599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.368748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.368828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.368980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.369068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.369223 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.369303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.369508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.369588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.connection = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.369749 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.369844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.connection = **** {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.369996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.370086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.370233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.370314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.370466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.370545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.370700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.370779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.370934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.371014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.371190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.371270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.371425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.371503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.371662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.371744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.371900 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.371980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.372133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.372212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.372365 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.372445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.372597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.372675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.372834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.372914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.373068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.373149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.373299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.373377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.373542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.373621 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.373778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.373858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.373938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80352) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 05 14:59:20.374014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 05 14:59:20.374171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80351) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 05 14:59:20.404638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.404822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.404911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.404990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.405094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.405216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.413404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.413494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.442033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.442185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.442271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.442416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.585190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.586145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.632964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.633198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.633920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.634157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.697426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.697652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.697820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.697924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.698051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.698137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.698274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.698367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.698487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.698578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.698701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.698915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.724172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.724290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.724391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.724498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.724609 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.724727 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.724835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.724951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.725036 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.725170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.725254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.725386 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.725468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.725603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.736908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.737024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.749638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.749934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.750151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.750394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.750524 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.750612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.750691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.752391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.752483 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.752560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.752635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.752708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.752792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80351) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.752869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80352) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 05 14:59:20.770231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.770444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 14:59:20.770777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.770867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.770960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 14:59:20.771062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 14:59:20.771255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.771401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.771503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.771595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 05 14:59:20.771757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.771840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.772010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.772092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.772344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.772597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.782739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.783015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.783306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.783390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.783588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.783673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.783900 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.784180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.791826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.792442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.795095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.795376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.795657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.795739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.795938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.796034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.796221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.796304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.796480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.796563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.796751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.796851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.797019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.797109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.797283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.797374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.797548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.797638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.797839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.797929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.798107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.798371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.824552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.825168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.826153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.826428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.826771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.827072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.832505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.832784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.833173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.833262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.833448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.833843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.840471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.840710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.841095 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.841375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.845204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.845492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.845773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.845862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.846053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.846144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.846326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.846417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.846598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.846698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.846865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.846956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.847159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.847250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.847437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.847699 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.865921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.866191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.866284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.866467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.870740 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.871012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.874365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.874638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.874729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.874930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.880371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.880567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.884010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.884178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.887625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.887806 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.887926 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.888102 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.888217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.888387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.888509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.888682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.888804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.888959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.889083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.889235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.908608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.908759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.908889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.909039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.909158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.909302 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.909418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.909560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.917284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.917456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.918989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.919150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.919290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.919375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.922512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.922604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.922815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.922898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.923110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.923194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.923397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.923481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.923684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.923767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.936805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.936915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.937084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.937177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.937362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.937455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.937642 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.937725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.937918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.937999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.939958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.940061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.940247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.940330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.958306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.958520 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.961575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.961787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.964902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.965102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.966622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.966818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.968292 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.968479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.968562 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.968758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.968847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.969049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.976675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.976836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.978271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.978434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.981509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.981674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.981804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.981963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.982198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.982349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.982482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.982626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.991608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.991743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:20.991895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.992034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.992185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.992314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:20.999982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80352) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:21.000138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80351) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 05 14:59:21.000258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.000347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.000532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.000617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.000828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.000911 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.001099 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.001185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.001520 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.001609 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.001798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.001891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.002066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.002147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.002335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.002466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.002604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.002687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.002877 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.002958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.003162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.003249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.003429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.003510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.003690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80352) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.003783 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80351) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 05 14:59:21.041480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5583280809c0 pid: 80351 (default app) Jul 05 14:59:21.041713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5583280809c0 pid: 80352 (default app) Jul 05 14:59:21.044837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636efdc-305d-4f9b-970a-a77808fcb91f None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:21.045131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636efdc-305d-4f9b-970a-a77808fcb91f None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:21.045411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636efdc-305d-4f9b-970a-a77808fcb91f None None] PATH_INFO: `/v3` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:21.050156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Tue Jul 5 14:59:21 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:28.624686 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558efe5e-9906-446d-a37a-7a1c384ee62f None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:28.625215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558efe5e-9906-446d-a37a-7a1c384ee62f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:28.625399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558efe5e-9906-446d-a37a-7a1c384ee62f None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:28.630146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:28.635285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:28.635657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:28.635848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:28.704838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: INFO dbcounter [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] Registered counter for database keystone Jul 05 14:59:28.725401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG oslo_db.sqlalchemy.engines [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80351) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 14:59:28.731656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writer thread running {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 14:59:28.911305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG passlib.handlers.bcrypt [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80351) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 05 14:59:28.911832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG passlib.handlers.bcrypt [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80351) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 05 14:59:28.968422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:28.970562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:29.096863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 05 14:59:29.249578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558efe5e-9906-446d-a37a-7a1c384ee62f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:29.320618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: INFO dbcounter [None req-558efe5e-9906-446d-a37a-7a1c384ee62f None None] Registered counter for database keystone Jul 05 14:59:29.340420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG oslo_db.sqlalchemy.engines [None req-558efe5e-9906-446d-a37a-7a1c384ee62f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80352) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 05 14:59:29.346844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writer thread running {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 05 14:59:29.465940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558efe5e-9906-446d-a37a-7a1c384ee62f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:29.470168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b3c12c-4ca0-459b-af24-1fb159f52984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:29.470835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3c12c-4ca0-459b-af24-1fb159f52984 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:29.471147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3c12c-4ca0-459b-af24-1fb159f52984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:29.471429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b3c12c-4ca0-459b-af24-1fb159f52984 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:29.479415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3c12c-4ca0-459b-af24-1fb159f52984 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:29.531543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f6b3c12c-4ca0-459b-af24-1fb159f52984 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 05 14:59:29.531900 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b3c12c-4ca0-459b-af24-1fb159f52984 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:29.541403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Tue Jul 5 14:59:29 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 14:59:29.548159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:29.562461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70e4878-d5a2-4905-842e-d18d6d8e5ce7 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:29.567496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24228e22-ee91-4e9f-91c6-7801de353a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:29.568613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24228e22-ee91-4e9f-91c6-7801de353a7f admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:29.568902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24228e22-ee91-4e9f-91c6-7801de353a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:29.569183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24228e22-ee91-4e9f-91c6-7801de353a7f admin admin] PATH_INFO: `/v3/projects/77588a8e554a4d32b7cfdcc698266c19` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:29.573270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24228e22-ee91-4e9f-91c6-7801de353a7f admin admin] RBAC: Authorizing `identity:get_project(project_id=77588a8e554a4d32b7cfdcc698266c19)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:29.623889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: WARNING keystone.common.rbac_enforcer.enforcer [None req-24228e22-ee91-4e9f-91c6-7801de353a7f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 05 14:59:29.624241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24228e22-ee91-4e9f-91c6-7801de353a7f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:29.629407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Tue Jul 5 14:59:29 2022] GET /identity/v3/projects/77588a8e554a4d32b7cfdcc698266c19 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:31.609800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d46051-3428-4aea-a4af-00fdcbe6c183 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:31.610299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d46051-3428-4aea-a4af-00fdcbe6c183 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:31.610299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d46051-3428-4aea-a4af-00fdcbe6c183 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:31.614424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:31.619734 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b60e1-19de-406f-b4c6-e9962e151cc8 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:31.620101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b60e1-19de-406f-b4c6-e9962e151cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:31.620299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b60e1-19de-406f-b4c6-e9962e151cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:31.638517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-505b60e1-19de-406f-b4c6-e9962e151cc8 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:31.640047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-505b60e1-19de-406f-b4c6-e9962e151cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:31.648256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 05 14:59:31.671183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d46051-3428-4aea-a4af-00fdcbe6c183 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:31.682959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d46051-3428-4aea-a4af-00fdcbe6c183 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:31.687176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cb7f2-3425-422d-9338-a656df56eb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:31.687858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb7f2-3425-422d-9338-a656df56eb58 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:31.688150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb7f2-3425-422d-9338-a656df56eb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:31.688433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb7f2-3425-422d-9338-a656df56eb58 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:31.696052 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cb7f2-3425-422d-9338-a656df56eb58 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:31.697836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cb7f2-3425-422d-9338-a656df56eb58 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:31.707247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Tue Jul 5 14:59:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 14:59:31.712261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b60e1-19de-406f-b4c6-e9962e151cc8 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:31.723972 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b60e1-19de-406f-b4c6-e9962e151cc8 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:31.728292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb95a4-dff5-4001-957c-cf810eb6f07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:31.729150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb95a4-dff5-4001-957c-cf810eb6f07a admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:31.729436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb95a4-dff5-4001-957c-cf810eb6f07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:31.729716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb95a4-dff5-4001-957c-cf810eb6f07a admin admin] PATH_INFO: `/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:31.731108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb95a4-dff5-4001-957c-cf810eb6f07a admin admin] RBAC: Authorizing `identity:get_user(user_id=7b7d0fd6414b417ab8bf3499e0f4961b)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:31.732934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb95a4-dff5-4001-957c-cf810eb6f07a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:31.734983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Tue Jul 5 14:59:31 2022] GET /identity/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:33.787764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340efd5-9d95-42e5-a614-a7b22e055a55 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:33.788308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340efd5-9d95-42e5-a614-a7b22e055a55 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:33.788308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b340efd5-9d95-42e5-a614-a7b22e055a55 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:33.789717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:33.794758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cd000-44e5-4cdc-bafe-e28fee2230f6 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:33.795151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cd000-44e5-4cdc-bafe-e28fee2230f6 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:33.795363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cd000-44e5-4cdc-bafe-e28fee2230f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:33.812217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-406cd000-44e5-4cdc-bafe-e28fee2230f6 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:33.814030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-406cd000-44e5-4cdc-bafe-e28fee2230f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:33.848619 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f818-7a51-405a-9adb-060be1738716 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:33.848918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f818-7a51-405a-9adb-060be1738716 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:33.849202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f818-7a51-405a-9adb-060be1738716 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:33.850760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:33.855580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb749170-9e71-44e3-8670-7204dff52b88 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:33.855880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb749170-9e71-44e3-8670-7204dff52b88 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:33.856161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb749170-9e71-44e3-8670-7204dff52b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:33.894165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:33.895969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG passlib.handlers.bcrypt [None req-fb749170-9e71-44e3-8670-7204dff52b88 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80352) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 05 14:59:33.896391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG passlib.handlers.bcrypt [None req-fb749170-9e71-44e3-8670-7204dff52b88 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80352) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 05 14:59:33.916381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cd000-44e5-4cdc-bafe-e28fee2230f6 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:33.939745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:33.942399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fb749170-9e71-44e3-8670-7204dff52b88 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:33.944023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fb749170-9e71-44e3-8670-7204dff52b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:34.017922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cd000-44e5-4cdc-bafe-e28fee2230f6 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:34.041798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 14:59:34.051198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfc88d-fa61-4274-bee9-966b8c1f19f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:34.052003 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfc88d-fa61-4274-bee9-966b8c1f19f5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:34.052290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfc88d-fa61-4274-bee9-966b8c1f19f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:34.052573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfc88d-fa61-4274-bee9-966b8c1f19f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:34.063650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfc88d-fa61-4274-bee9-966b8c1f19f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:34.064241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb749170-9e71-44e3-8670-7204dff52b88 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-13dfc88d-fa61-4274-bee9-966b8c1f19f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:34.074879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:34.075895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 14:59:34.076746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 14:59:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 14:59:34.082475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfc88d-fa61-4274-bee9-966b8c1f19f5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:34.162169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:34.162611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb749170-9e71-44e3-8670-7204dff52b88 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:34.165012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:34.181490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfc88d-fa61-4274-bee9-966b8c1f19f5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:34.201219 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811b113-da69-4d24-bfdd-bfc6f3b3f250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:34.202133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811b113-da69-4d24-bfdd-bfc6f3b3f250 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:34.202423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811b113-da69-4d24-bfdd-bfc6f3b3f250 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:34.202962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811b113-da69-4d24-bfdd-bfc6f3b3f250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:34.220702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc492c-ce02-4b3b-abb6-0a04d52b4903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:34.221373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc492c-ce02-4b3b-abb6-0a04d52b4903 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:34.221670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc492c-ce02-4b3b-abb6-0a04d52b4903 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:34.221938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc492c-ce02-4b3b-abb6-0a04d52b4903 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:34.222704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc492c-ce02-4b3b-abb6-0a04d52b4903 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:34.223014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc492c-ce02-4b3b-abb6-0a04d52b4903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:34.224455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc492c-ce02-4b3b-abb6-0a04d52b4903 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:34.236017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:34.241362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc492c-ce02-4b3b-abb6-0a04d52b4903 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:34.271887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811b113-da69-4d24-bfdd-bfc6f3b3f250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:34.273417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7811b113-da69-4d24-bfdd-bfc6f3b3f250 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:34.338719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc492c-ce02-4b3b-abb6-0a04d52b4903 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:34.368816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 8/16] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 14:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 14:59:34.371658 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac77b0-eebd-426b-b066-73ac2b2e6f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:34.372525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac77b0-eebd-426b-b066-73ac2b2e6f37 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:34.372815 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac77b0-eebd-426b-b066-73ac2b2e6f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:34.373213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cac77b0-eebd-426b-b066-73ac2b2e6f37 None admin] PATH_INFO: `/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:34.374272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811b113-da69-4d24-bfdd-bfc6f3b3f250 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:34.374403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac77b0-eebd-426b-b066-73ac2b2e6f37 None admin] RBAC: Authorizing `identity:get_user(user_id=7b7d0fd6414b417ab8bf3499e0f4961b)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:34.376412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cac77b0-eebd-426b-b066-73ac2b2e6f37 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:34.377542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:34.382547 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:34 2022] GET /identity/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:34.383654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac77b0-eebd-426b-b066-73ac2b2e6f37 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:34.397761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:34.471968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7811b113-da69-4d24-bfdd-bfc6f3b3f250 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:34.483405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cac77b0-eebd-426b-b066-73ac2b2e6f37 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:34.503863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfd280-7015-41f4-87f3-e84c79c220a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:34.504662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfd280-7015-41f4-87f3-e84c79c220a8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:34.504950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfd280-7015-41f4-87f3-e84c79c220a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:34.505233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfd280-7015-41f4-87f3-e84c79c220a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:34.518666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9b455-98da-45f4-bc2b-7cf6a9e7f15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:34.519486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9b455-98da-45f4-bc2b-7cf6a9e7f15a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:34.519769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9b455-98da-45f4-bc2b-7cf6a9e7f15a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:34.520046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9b455-98da-45f4-bc2b-7cf6a9e7f15a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:34.521216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9b455-98da-45f4-bc2b-7cf6a9e7f15a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:34.523181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a9b455-98da-45f4-bc2b-7cf6a9e7f15a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:34.523798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfd280-7015-41f4-87f3-e84c79c220a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:34.524766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 10/18] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:34.525262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dfd280-7015-41f4-87f3-e84c79c220a8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:34.530360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9b455-98da-45f4-bc2b-7cf6a9e7f15a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-21dfd280-7015-41f4-87f3-e84c79c220a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 14:59:34.546742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:34.547809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:34.548632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 14:59:34.548632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 14:59:34.548632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 14:59:34.548632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:34.548632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 9/19] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 14:59:34.553524 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfd280-7015-41f4-87f3-e84c79c220a8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:34.608552 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:34.611523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:34.630838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a9b455-98da-45f4-bc2b-7cf6a9e7f15a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:34.652779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dfd280-7015-41f4-87f3-e84c79c220a8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:34.664164 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c060a8c-2e5a-442a-97c5-9eacbc6ac9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:34.664817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c060a8c-2e5a-442a-97c5-9eacbc6ac9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:34.665104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c060a8c-2e5a-442a-97c5-9eacbc6ac9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:34.665382 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c060a8c-2e5a-442a-97c5-9eacbc6ac9b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:34.666220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c060a8c-2e5a-442a-97c5-9eacbc6ac9b8 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.domain.id=default, user.id=7b7d0fd6414b417ab8bf3499e0f4961b) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:34.666540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c060a8c-2e5a-442a-97c5-9eacbc6ac9b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:34.668020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c060a8c-2e5a-442a-97c5-9eacbc6ac9b8 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:34.674763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1322 bytes} [Tue Jul 5 14:59:34 2022] GET /identity/v3/role_assignments?user.id=7b7d0fd6414b417ab8bf3499e0f4961b&scope.domain.id=default&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:34.686240 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b039b0a-1728-4f4a-9abf-1f8fc772fce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:34.686931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b039b0a-1728-4f4a-9abf-1f8fc772fce4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:34.687252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b039b0a-1728-4f4a-9abf-1f8fc772fce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:34.687539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b039b0a-1728-4f4a-9abf-1f8fc772fce4 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:34.688310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b039b0a-1728-4f4a-9abf-1f8fc772fce4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:34.688624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b039b0a-1728-4f4a-9abf-1f8fc772fce4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:34.690089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b039b0a-1728-4f4a-9abf-1f8fc772fce4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:34.710563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 14:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:37.013645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65195f24-357f-4fba-bf47-ab41565a8e07 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:37.014638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65195f24-357f-4fba-bf47-ab41565a8e07 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:37.015214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65195f24-357f-4fba-bf47-ab41565a8e07 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:37.017303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:37.022638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7659bb-1563-4234-a4b7-82be3fafdc1a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:37.023464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7659bb-1563-4234-a4b7-82be3fafdc1a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:37.023943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7659bb-1563-4234-a4b7-82be3fafdc1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:37.045393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3d7659bb-1563-4234-a4b7-82be3fafdc1a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:37.047481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3d7659bb-1563-4234-a4b7-82be3fafdc1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:37.140957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:37.173834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65195f24-357f-4fba-bf47-ab41565a8e07 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:37.306449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9b31d-44f5-43e6-a915-59c225910906 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:37.307106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9b31d-44f5-43e6-a915-59c225910906 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:37.307589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9b31d-44f5-43e6-a915-59c225910906 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:37.309401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:37.313826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65195f24-357f-4fba-bf47-ab41565a8e07 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:37.315347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48767c-bf8a-4565-b764-be6e9336be85 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:37.315856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48767c-bf8a-4565-b764-be6e9336be85 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:37.316336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b48767c-bf8a-4565-b764-be6e9336be85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:37.356132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3b48767c-bf8a-4565-b764-be6e9336be85 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:37.357693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3b48767c-bf8a-4565-b764-be6e9336be85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:37.364744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0492b998-a7ea-4fef-98c0-02ae2f277f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:37.365790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492b998-a7ea-4fef-98c0-02ae2f277f73 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:37.366229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492b998-a7ea-4fef-98c0-02ae2f277f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:37.366657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0492b998-a7ea-4fef-98c0-02ae2f277f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:37.379437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492b998-a7ea-4fef-98c0-02ae2f277f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:37.381240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0492b998-a7ea-4fef-98c0-02ae2f277f73 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:37.388744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-0492b998-a7ea-4fef-98c0-02ae2f277f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 14:59:37.405279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:37.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:37.408233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 14:59:37.408233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 14:59:37.408233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 14:59:37.408233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 14:59:37.408233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 13/25] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:37.412084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c3b3-1839-4895-a30c-7c53a4795766 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:37.412084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c3b3-1839-4895-a30c-7c53a4795766 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:37.412084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346c3b3-1839-4895-a30c-7c53a4795766 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:37.412084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 14/26] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:37.413684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346c3b3-1839-4895-a30c-7c53a4795766 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:37.497338 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:37.530921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346c3b3-1839-4895-a30c-7c53a4795766 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:37.570395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118493b-1839-4b66-8ffc-69ab6b248543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:37.570992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118493b-1839-4b66-8ffc-69ab6b248543 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:37.571447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118493b-1839-4b66-8ffc-69ab6b248543 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:37.571737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118493b-1839-4b66-8ffc-69ab6b248543 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:37.572479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118493b-1839-4b66-8ffc-69ab6b248543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:37.572791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118493b-1839-4b66-8ffc-69ab6b248543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:37.574201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118493b-1839-4b66-8ffc-69ab6b248543 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:37.574850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 13/27] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:37.576169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2004c61-50db-4ffc-807c-b7f246a8909b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:37.576464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2004c61-50db-4ffc-807c-b7f246a8909b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:37.576748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2004c61-50db-4ffc-807c-b7f246a8909b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:37.599602 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 15/28] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 14:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:37.600891 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118493b-1839-4b66-8ffc-69ab6b248543 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:37.616840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e2004c61-50db-4ffc-807c-b7f246a8909b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:37.618058 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e2004c61-50db-4ffc-807c-b7f246a8909b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:37.671007 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:37.722491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118493b-1839-4b66-8ffc-69ab6b248543 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:37.762557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ef2bd-16c9-4a74-a2cf-2a29e668fb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:37.763401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef2bd-16c9-4a74-a2cf-2a29e668fb8d None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:37.763695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef2bd-16c9-4a74-a2cf-2a29e668fb8d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:37.764237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ef2bd-16c9-4a74-a2cf-2a29e668fb8d None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:37.764979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ef2bd-16c9-4a74-a2cf-2a29e668fb8d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:37.766992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ef2bd-16c9-4a74-a2cf-2a29e668fb8d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:37.777533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 14/29] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:37.778429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2004c61-50db-4ffc-807c-b7f246a8909b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:37.910747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2004c61-50db-4ffc-807c-b7f246a8909b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:37.918966 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:37.947935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1cc5b-0b74-4d73-ad0c-e96d8c9d135d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:37.948726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1cc5b-0b74-4d73-ad0c-e96d8c9d135d None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:37.949084 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1cc5b-0b74-4d73-ad0c-e96d8c9d135d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:37.949437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1cc5b-0b74-4d73-ad0c-e96d8c9d135d None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:37.950390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1cc5b-0b74-4d73-ad0c-e96d8c9d135d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:37.951205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 16/30] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 14:59:37 2022] POST /identity/v3/projects => generated 301 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 14:59:37.952106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1cc5b-0b74-4d73-ad0c-e96d8c9d135d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:37.952394 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ef2bd-16c9-4a74-a2cf-2a29e668fb8d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:38.085767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 15/31] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jul 5 14:59:37 2022] POST /identity/v3/roles => generated 219 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 14:59:38.095599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e621-e525-4e2c-90b2-ca29618e28a8 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.095999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e621-e525-4e2c-90b2-ca29618e28a8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.096409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e621-e525-4e2c-90b2-ca29618e28a8 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.098337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 16/32] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:38 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:38.107996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831c89b-c1a2-4017-a292-50808e20424d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.108286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831c89b-c1a2-4017-a292-50808e20424d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.108568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831c89b-c1a2-4017-a292-50808e20424d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.123692 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ef2bd-16c9-4a74-a2cf-2a29e668fb8d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:38.135781 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:38.151740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b831c89b-c1a2-4017-a292-50808e20424d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:38.153585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b831c89b-c1a2-4017-a292-50808e20424d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:38.188047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cfe91-58d4-4ff8-88ab-b218bdae440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:38.188804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cfe91-58d4-4ff8-88ab-b218bdae440a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.189096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cfe91-58d4-4ff8-88ab-b218bdae440a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.189374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cfe91-58d4-4ff8-88ab-b218bdae440a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.212697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1cfe91-58d4-4ff8-88ab-b218bdae440a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-9c1cfe91-58d4-4ff8-88ab-b218bdae440a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:38.222114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:38.223221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 14:59:38.224237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 17/33] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 14:59:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:38.231811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cfe91-58d4-4ff8-88ab-b218bdae440a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:38.313364 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 17/34] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 05 14:59:38.343021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cfe91-58d4-4ff8-88ab-b218bdae440a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:38.348100 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831c89b-c1a2-4017-a292-50808e20424d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:38.380540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883cfce-f732-495c-914a-f575fcb686df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:38.381228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883cfce-f732-495c-914a-f575fcb686df None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.381513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883cfce-f732-495c-914a-f575fcb686df None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.381798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883cfce-f732-495c-914a-f575fcb686df None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.382573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883cfce-f732-495c-914a-f575fcb686df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:38.382885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883cfce-f732-495c-914a-f575fcb686df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:38.384591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883cfce-f732-495c-914a-f575fcb686df None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:38.397237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 18/35] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:38.400831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883cfce-f732-495c-914a-f575fcb686df None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:38.435407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:38.509828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:38.520885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831c89b-c1a2-4017-a292-50808e20424d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:38.533357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883cfce-f732-495c-914a-f575fcb686df None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:38.563262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9cb3fb-cf19-4349-af56-56044fb990a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:38.564236 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9cb3fb-cf19-4349-af56-56044fb990a2 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.564716 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9cb3fb-cf19-4349-af56-56044fb990a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.565295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9cb3fb-cf19-4349-af56-56044fb990a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.566347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9cb3fb-cf19-4349-af56-56044fb990a2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:38.568102 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9cb3fb-cf19-4349-af56-56044fb990a2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:38.578977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28358b67-b28d-4b10-af27-0211124cc586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:38.580020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28358b67-b28d-4b10-af27-0211124cc586 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.580411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28358b67-b28d-4b10-af27-0211124cc586 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.580695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28358b67-b28d-4b10-af27-0211124cc586 None admin] PATH_INFO: `/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.582013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28358b67-b28d-4b10-af27-0211124cc586 None admin] RBAC: Authorizing `identity:get_user(user_id=7b7d0fd6414b417ab8bf3499e0f4961b)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:38.583588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28358b67-b28d-4b10-af27-0211124cc586 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:38.584088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 18/36] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jul 5 14:59:38 2022] POST /identity/v3/roles => generated 225 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 14:59:38.585282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 19/37] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:38 2022] GET /identity/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:38.585680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ac18-c1f3-4a5a-b6c0-ccb48473756f None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.585979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ac18-c1f3-4a5a-b6c0-ccb48473756f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.586314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ac18-c1f3-4a5a-b6c0-ccb48473756f None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.587731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 19/38] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:38.590846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28358b67-b28d-4b10-af27-0211124cc586 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:38.596060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833f633-4195-4d5b-b0a0-2571bf928906 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.598157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833f633-4195-4d5b-b0a0-2571bf928906 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.598157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833f633-4195-4d5b-b0a0-2571bf928906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.616566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5833f633-4195-4d5b-b0a0-2571bf928906 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:38.617856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5833f633-4195-4d5b-b0a0-2571bf928906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:38.699755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 20/39] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 14:59:38.701174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9ff45-e95d-4e27-9710-4a327003cb4a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.701464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9ff45-e95d-4e27-9710-4a327003cb4a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.701739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f9ff45-e95d-4e27-9710-4a327003cb4a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.704068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 21/40] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:38.711652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef5d2b-4ebd-4131-a6f0-3083eb4811bb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.711944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef5d2b-4ebd-4131-a6f0-3083eb4811bb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.712226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef5d2b-4ebd-4131-a6f0-3083eb4811bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.726005 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:38.731428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28358b67-b28d-4b10-af27-0211124cc586 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:38.732207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e9ef5d2b-4ebd-4131-a6f0-3083eb4811bb None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:38.739948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e9ef5d2b-4ebd-4131-a6f0-3083eb4811bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:38.778101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687097a3-9b84-4b69-bd97-74f279c205cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:38.778283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687097a3-9b84-4b69-bd97-74f279c205cc None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.778569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687097a3-9b84-4b69-bd97-74f279c205cc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.778845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687097a3-9b84-4b69-bd97-74f279c205cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.780187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687097a3-9b84-4b69-bd97-74f279c205cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:38.781650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687097a3-9b84-4b69-bd97-74f279c205cc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:38.784608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 20/41] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:38.784608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687097a3-9b84-4b69-bd97-74f279c205cc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:38.822882 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 22/42] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 14:59:38.823788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef5d2b-4ebd-4131-a6f0-3083eb4811bb None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:38.846284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:38.934096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef5d2b-4ebd-4131-a6f0-3083eb4811bb None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:38.968767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f91358-1049-4798-adc1-ba5af1916952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:38.969645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f91358-1049-4798-adc1-ba5af1916952 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:38.969931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f91358-1049-4798-adc1-ba5af1916952 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:38.970214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f91358-1049-4798-adc1-ba5af1916952 None admin] PATH_INFO: `/v3/domains/default/users/7b7d0fd6414b417ab8bf3499e0f4961b/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:38.972643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f91358-1049-4798-adc1-ba5af1916952 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7b7d0fd6414b417ab8bf3499e0f4961b, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:38.974094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f91358-1049-4798-adc1-ba5af1916952 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:38.989326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 23/43] 10.222.0.23 () {58 vars in 1361 bytes} [Tue Jul 5 14:59:38 2022] PUT /identity/v3/domains/default/users/7b7d0fd6414b417ab8bf3499e0f4961b/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 14:59:38.990134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f91358-1049-4798-adc1-ba5af1916952 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:39.018596 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687097a3-9b84-4b69-bd97-74f279c205cc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:39.063334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9fa66-8b4c-4adc-a320-b0f50f4db683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:39.063936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9fa66-8b4c-4adc-a320-b0f50f4db683 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:39.064234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9fa66-8b4c-4adc-a320-b0f50f4db683 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:39.064511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9fa66-8b4c-4adc-a320-b0f50f4db683 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:39.065287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9fa66-8b4c-4adc-a320-b0f50f4db683 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:39.066830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9fa66-8b4c-4adc-a320-b0f50f4db683 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:39.083641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 21/44] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jul 5 14:59:38 2022] POST /identity/v3/roles => generated 223 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 14:59:39.095243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:39.120429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f91358-1049-4798-adc1-ba5af1916952 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:39.165193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58cbdd1-8f0d-4ead-bc03-68b89104e676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:39.166017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cbdd1-8f0d-4ead-bc03-68b89104e676 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:39.166301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cbdd1-8f0d-4ead-bc03-68b89104e676 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:39.166581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cbdd1-8f0d-4ead-bc03-68b89104e676 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:39.167897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cbdd1-8f0d-4ead-bc03-68b89104e676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:39.169924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cbdd1-8f0d-4ead-bc03-68b89104e676 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:39.172620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 24/45] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:39.178347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9fa66-8b4c-4adc-a320-b0f50f4db683 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:39.211187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:39.295192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9fa66-8b4c-4adc-a320-b0f50f4db683 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:39.331206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f811d2-8fcb-4ba4-bff7-cc48fca2dcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:39.331801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f811d2-8fcb-4ba4-bff7-cc48fca2dcb1 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:39.332092 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f811d2-8fcb-4ba4-bff7-cc48fca2dcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:39.332379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f811d2-8fcb-4ba4-bff7-cc48fca2dcb1 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:39.333111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f811d2-8fcb-4ba4-bff7-cc48fca2dcb1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:39.334581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f811d2-8fcb-4ba4-bff7-cc48fca2dcb1 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:39.360318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 22/46] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 14:59:39 2022] POST /identity/v3/projects => generated 312 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 14:59:41.050660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a270f17-0842-457c-86db-5d43bdf89e2c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:41.051227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a270f17-0842-457c-86db-5d43bdf89e2c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:41.051227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a270f17-0842-457c-86db-5d43bdf89e2c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:41.052626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 25/47] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:41.057842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c5450-87a0-4c7d-be7e-0d817588e5d3 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:41.058218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c5450-87a0-4c7d-be7e-0d817588e5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:41.058424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c5450-87a0-4c7d-be7e-0d817588e5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:41.075243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-1a4c5450-87a0-4c7d-be7e-0d817588e5d3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:41.076767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-1a4c5450-87a0-4c7d-be7e-0d817588e5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:41.143098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 23/48] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:41.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a270f17-0842-457c-86db-5d43bdf89e2c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:41.276795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a270f17-0842-457c-86db-5d43bdf89e2c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:41.310049 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc021e1-70da-4b62-99e0-2bd94ea7ddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:41.310767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc021e1-70da-4b62-99e0-2bd94ea7ddab None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:41.311085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc021e1-70da-4b62-99e0-2bd94ea7ddab None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:41.311375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc021e1-70da-4b62-99e0-2bd94ea7ddab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:41.322351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc021e1-70da-4b62-99e0-2bd94ea7ddab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4fc021e1-70da-4b62-99e0-2bd94ea7ddab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:41.331913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:41.332911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:41.333785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 26/49] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 14:59:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:41.338881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c5450-87a0-4c7d-be7e-0d817588e5d3 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:41.349565 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:41.383675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab1e90-aff1-4d82-964a-34765e4aeee2 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:41.383965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab1e90-aff1-4d82-964a-34765e4aeee2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:41.384247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab1e90-aff1-4d82-964a-34765e4aeee2 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:41.385622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 27/50] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 14:59:41.391145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96bf21-7c38-437a-b1dc-e9f0dbb36273 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:41.391439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96bf21-7c38-437a-b1dc-e9f0dbb36273 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:41.391721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96bf21-7c38-437a-b1dc-e9f0dbb36273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:41.398110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:41.409513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ab96bf21-7c38-437a-b1dc-e9f0dbb36273 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:41.410732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ab96bf21-7c38-437a-b1dc-e9f0dbb36273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:41.438822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c5450-87a0-4c7d-be7e-0d817588e5d3 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:41.473512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecc8ef-18d2-4b5b-84d2-b4e1326297b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:41.474194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecc8ef-18d2-4b5b-84d2-b4e1326297b4 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:41.474489 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecc8ef-18d2-4b5b-84d2-b4e1326297b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:41.474770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecc8ef-18d2-4b5b-84d2-b4e1326297b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:41.475572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecc8ef-18d2-4b5b-84d2-b4e1326297b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:41.475886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecc8ef-18d2-4b5b-84d2-b4e1326297b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:41.476400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 28/51] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 14:59:41.477298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecc8ef-18d2-4b5b-84d2-b4e1326297b4 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:41.485448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 24/52] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:41.490745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96bf21-7c38-437a-b1dc-e9f0dbb36273 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:41.499303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecc8ef-18d2-4b5b-84d2-b4e1326297b4 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:41.569976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:41.590639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96bf21-7c38-437a-b1dc-e9f0dbb36273 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:41.601326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecc8ef-18d2-4b5b-84d2-b4e1326297b4 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:41.637886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61c0c-a4b4-45d2-b91c-d7acf854a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:41.638710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61c0c-a4b4-45d2-b91c-d7acf854a952 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:41.639009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61c0c-a4b4-45d2-b91c-d7acf854a952 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:41.639327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b61c0c-a4b4-45d2-b91c-d7acf854a952 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:41.640641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b61c0c-a4b4-45d2-b91c-d7acf854a952 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:41.642167 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b61c0c-a4b4-45d2-b91c-d7acf854a952 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:41.643914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 25/53] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:41.649761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61c0c-a4b4-45d2-b91c-d7acf854a952 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:41.728940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88532612-a480-4a65-985d-9c6bcb379eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:41.729806 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88532612-a480-4a65-985d-9c6bcb379eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:41.730088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88532612-a480-4a65-985d-9c6bcb379eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:41.730366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88532612-a480-4a65-985d-9c6bcb379eb1 None admin] PATH_INFO: `/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:41.731590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88532612-a480-4a65-985d-9c6bcb379eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=7b7d0fd6414b417ab8bf3499e0f4961b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:41.733041 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88532612-a480-4a65-985d-9c6bcb379eb1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:41.735245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 29/54] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:41 2022] GET /identity/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:41.740497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88532612-a480-4a65-985d-9c6bcb379eb1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:41.742994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:41.753416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b61c0c-a4b4-45d2-b91c-d7acf854a952 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:41.786258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a88696-00d0-4bb2-ab9a-c131f238d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:41.786925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a88696-00d0-4bb2-ab9a-c131f238d880 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:41.787265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a88696-00d0-4bb2-ab9a-c131f238d880 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:41.787544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a88696-00d0-4bb2-ab9a-c131f238d880 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:41.788259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a88696-00d0-4bb2-ab9a-c131f238d880 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:41.789727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a88696-00d0-4bb2-ab9a-c131f238d880 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:41.798555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:41.816119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 26/55] 10.222.0.23 () {60 vars in 1140 bytes} [Tue Jul 5 14:59:41 2022] POST /identity/v3/projects => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 14:59:41.840530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88532612-a480-4a65-985d-9c6bcb379eb1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:41.877681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc63ff2-bbfe-45ce-a2f6-bde8ebbc78d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:41.878512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc63ff2-bbfe-45ce-a2f6-bde8ebbc78d7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:41.878804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc63ff2-bbfe-45ce-a2f6-bde8ebbc78d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:41.879125 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc63ff2-bbfe-45ce-a2f6-bde8ebbc78d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:41.880402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc63ff2-bbfe-45ce-a2f6-bde8ebbc78d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:41.881894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc63ff2-bbfe-45ce-a2f6-bde8ebbc78d7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:41.883627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 30/56] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:41.889757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a88696-00d0-4bb2-ab9a-c131f238d880 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:41.996844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a88696-00d0-4bb2-ab9a-c131f238d880 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:42.029919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819fd4a-ad1a-4e2a-9483-3b9ea1bc90b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:42.030586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819fd4a-ad1a-4e2a-9483-3b9ea1bc90b3 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:42.031449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819fd4a-ad1a-4e2a-9483-3b9ea1bc90b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:42.031751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819fd4a-ad1a-4e2a-9483-3b9ea1bc90b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:42.032573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819fd4a-ad1a-4e2a-9483-3b9ea1bc90b3 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.domain.id=default, user.id=7b7d0fd6414b417ab8bf3499e0f4961b) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:42.032886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819fd4a-ad1a-4e2a-9483-3b9ea1bc90b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:42.034329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819fd4a-ad1a-4e2a-9483-3b9ea1bc90b3 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:42.040202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 27/57] 10.222.0.23 () {56 vars in 1322 bytes} [Tue Jul 5 14:59:41 2022] GET /identity/v3/role_assignments?user.id=7b7d0fd6414b417ab8bf3499e0f4961b&scope.domain.id=default&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:43.824603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7e524-dea3-439c-81d6-4dcadb7da4a8 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:43.825112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7e524-dea3-439c-81d6-4dcadb7da4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:43.825211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7e524-dea3-439c-81d6-4dcadb7da4a8 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:43.826533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 31/58] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:43.832354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca143d6-408c-47f4-a4a4-6caaf30c58fe None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:43.832712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca143d6-408c-47f4-a4a4-6caaf30c58fe None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:43.832916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca143d6-408c-47f4-a4a4-6caaf30c58fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:43.849725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5ca143d6-408c-47f4-a4a4-6caaf30c58fe None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:43.850910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5ca143d6-408c-47f4-a4a4-6caaf30c58fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:43.898000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:43.916065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 28/59] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:43.940211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7e524-dea3-439c-81d6-4dcadb7da4a8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:44.007493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:44.039055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7e524-dea3-439c-81d6-4dcadb7da4a8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:44.071237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8798e-8388-4cc0-9d48-d70ab78920cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:44.072020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8798e-8388-4cc0-9d48-d70ab78920cd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:44.072308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8798e-8388-4cc0-9d48-d70ab78920cd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:44.072588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8798e-8388-4cc0-9d48-d70ab78920cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:44.073753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8798e-8388-4cc0-9d48-d70ab78920cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:44.075334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8798e-8388-4cc0-9d48-d70ab78920cd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:44.076880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 32/60] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:44.082361 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca143d6-408c-47f4-a4a4-6caaf30c58fe None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:44.176218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca143d6-408c-47f4-a4a4-6caaf30c58fe None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:44.209231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec948ff6-e9de-407f-93f7-d95e7ea8b929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:44.209908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec948ff6-e9de-407f-93f7-d95e7ea8b929 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:44.210194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec948ff6-e9de-407f-93f7-d95e7ea8b929 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:44.210474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec948ff6-e9de-407f-93f7-d95e7ea8b929 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:44.211225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec948ff6-e9de-407f-93f7-d95e7ea8b929 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:44.212667 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec948ff6-e9de-407f-93f7-d95e7ea8b929 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:44.238110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 29/61] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 14:59:44 2022] POST /identity/v3/users => generated 275 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 14:59:47.382853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04a445-5656-4e7d-b085-8aa5ee83febe None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:47.383765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04a445-5656-4e7d-b085-8aa5ee83febe None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:47.384188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04a445-5656-4e7d-b085-8aa5ee83febe None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:47.385704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 33/62] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:47.391781 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe5be2-c785-4ecf-ad61-e8e6bb113e92 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:47.392176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe5be2-c785-4ecf-ad61-e8e6bb113e92 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:47.392573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe5be2-c785-4ecf-ad61-e8e6bb113e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:47.414754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-babe5be2-c785-4ecf-ad61-e8e6bb113e92 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:47.416011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-babe5be2-c785-4ecf-ad61-e8e6bb113e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:47.466131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:47.510886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 30/63] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:47.542310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04a445-5656-4e7d-b085-8aa5ee83febe None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:47.617314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f54c86-0a34-44be-89ab-7b4b0561e2e5 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:47.618014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f54c86-0a34-44be-89ab-7b4b0561e2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:47.618630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f54c86-0a34-44be-89ab-7b4b0561e2e5 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:47.622856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 31/64] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:47.629298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360446d6-0cc8-466b-ad9e-368a2fefc00d None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:47.629875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360446d6-0cc8-466b-ad9e-368a2fefc00d None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:47.630494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360446d6-0cc8-466b-ad9e-368a2fefc00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:47.660684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-360446d6-0cc8-466b-ad9e-368a2fefc00d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:47.661909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-360446d6-0cc8-466b-ad9e-368a2fefc00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:47.681740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04a445-5656-4e7d-b085-8aa5ee83febe None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:47.724733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e95da-be81-4d53-9b95-26aaa63387f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:47.725454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e95da-be81-4d53-9b95-26aaa63387f7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:47.725737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e95da-be81-4d53-9b95-26aaa63387f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:47.726374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e95da-be81-4d53-9b95-26aaa63387f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:47.735415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:47.740881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412e95da-be81-4d53-9b95-26aaa63387f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-412e95da-be81-4d53-9b95-26aaa63387f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:47.751948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:47.752898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 14:59:47.753867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:47.754452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 32/65] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 14:59:47.754452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5db01-29b8-43c6-92f4-36436fb69fa1 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:47.754452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5db01-29b8-43c6-92f4-36436fb69fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:47.754452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5db01-29b8-43c6-92f4-36436fb69fa1 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:47.755495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 33/66] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:47.756237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 34/67] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:47.763920 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5db01-29b8-43c6-92f4-36436fb69fa1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:47.767197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71b27b-135c-4586-8883-5f8fea0588a9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:47.767492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71b27b-135c-4586-8883-5f8fea0588a9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:47.767775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e71b27b-135c-4586-8883-5f8fea0588a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:47.786605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4e71b27b-135c-4586-8883-5f8fea0588a9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:47.788079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4e71b27b-135c-4586-8883-5f8fea0588a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:47.855974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 35/68] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 14:59:47.858288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71b27b-135c-4586-8883-5f8fea0588a9 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:47.873676 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:47.879679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5db01-29b8-43c6-92f4-36436fb69fa1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:47.928575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff497395-e7a2-4323-9afd-f6c4e176b4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:47.929629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff497395-e7a2-4323-9afd-f6c4e176b4aa None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:47.930354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff497395-e7a2-4323-9afd-f6c4e176b4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:47.930786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff497395-e7a2-4323-9afd-f6c4e176b4aa None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:47.931798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff497395-e7a2-4323-9afd-f6c4e176b4aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:47.932281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff497395-e7a2-4323-9afd-f6c4e176b4aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:47.933882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff497395-e7a2-4323-9afd-f6c4e176b4aa None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:47.944714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 34/69] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 14:59:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:47.945958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff497395-e7a2-4323-9afd-f6c4e176b4aa None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:47.963291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e71b27b-135c-4586-8883-5f8fea0588a9 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:47.999085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ce316-4747-428b-8b34-350835ac3081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:47.999949 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ce316-4747-428b-8b34-350835ac3081 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.000297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ce316-4747-428b-8b34-350835ac3081 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.000643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ce316-4747-428b-8b34-350835ac3081 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.005858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:48.012164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ce316-4747-428b-8b34-350835ac3081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8c4ce316-4747-428b-8b34-350835ac3081 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:48.021663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:48.022732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 14:59:48.023593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:48.024992 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 36/70] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 14:59:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:48.025811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ce316-4747-428b-8b34-350835ac3081 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:48.073202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff497395-e7a2-4323-9afd-f6c4e176b4aa None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.106034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e0a91c-1b0d-4b37-b97a-b7da0eb675f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.106849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0a91c-1b0d-4b37-b97a-b7da0eb675f0 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.107170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0a91c-1b0d-4b37-b97a-b7da0eb675f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.107458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e0a91c-1b0d-4b37-b97a-b7da0eb675f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.117893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e0a91c-1b0d-4b37-b97a-b7da0eb675f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:48.123110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ce316-4747-428b-8b34-350835ac3081 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-c3e0a91c-1b0d-4b37-b97a-b7da0eb675f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:48.126629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:48.127492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 14:59:48.128251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 35/71] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 14:59:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:48.129349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e319c9-9856-4a35-b573-282d66a4f544 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.129349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e319c9-9856-4a35-b573-282d66a4f544 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.129615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e319c9-9856-4a35-b573-282d66a4f544 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.130892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 36/72] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:48.132001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f3f65-2f64-4c9d-85ae-d5581845ddca None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.132294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f3f65-2f64-4c9d-85ae-d5581845ddca None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.132579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f3f65-2f64-4c9d-85ae-d5581845ddca None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.133797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 37/73] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:48.134669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f3f65-2f64-4c9d-85ae-d5581845ddca None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:48.158568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:48.161645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f91e26-97fc-4a4e-ad25-e98cf5563b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.162530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f91e26-97fc-4a4e-ad25-e98cf5563b50 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.162817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f91e26-97fc-4a4e-ad25-e98cf5563b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.163121 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f91e26-97fc-4a4e-ad25-e98cf5563b50 None admin] PATH_INFO: `/v3/users/49df8493343b42b0bb70939a8aa8aee1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.178387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f91e26-97fc-4a4e-ad25-e98cf5563b50 None admin] RBAC: Authorizing `identity:get_user(user_id=49df8493343b42b0bb70939a8aa8aee1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:48.179894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f91e26-97fc-4a4e-ad25-e98cf5563b50 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:48.181567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 37/74] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:48 2022] GET /identity/v3/users/49df8493343b42b0bb70939a8aa8aee1 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:48.182335 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f91e26-97fc-4a4e-ad25-e98cf5563b50 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:48.231474 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:48.240055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f3f65-2f64-4c9d-85ae-d5581845ddca None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.273271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8c8c1-54a1-495e-98dd-719a0caf621b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.273878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8c8c1-54a1-495e-98dd-719a0caf621b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.274168 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8c8c1-54a1-495e-98dd-719a0caf621b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.274451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8c8c1-54a1-495e-98dd-719a0caf621b None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.275231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8c8c1-54a1-495e-98dd-719a0caf621b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:48.275545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8c8c1-54a1-495e-98dd-719a0caf621b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:48.276927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8c8c1-54a1-495e-98dd-719a0caf621b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:48.284521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f91e26-97fc-4a4e-ad25-e98cf5563b50 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.285491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 38/75] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:48.286684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94189c8-7215-4c7a-ae77-7f2209c85c46 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.286976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94189c8-7215-4c7a-ae77-7f2209c85c46 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.287286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94189c8-7215-4c7a-ae77-7f2209c85c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.303098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c94189c8-7215-4c7a-ae77-7f2209c85c46 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:48.304303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c94189c8-7215-4c7a-ae77-7f2209c85c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:48.326204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5317cf9-d389-488f-b433-b90c038d8495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.326899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5317cf9-d389-488f-b433-b90c038d8495 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.327226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5317cf9-d389-488f-b433-b90c038d8495 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.327801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5317cf9-d389-488f-b433-b90c038d8495 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.328571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5317cf9-d389-488f-b433-b90c038d8495 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:48.328889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5317cf9-d389-488f-b433-b90c038d8495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:48.330286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5317cf9-d389-488f-b433-b90c038d8495 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:48.342261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 38/76] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 14:59:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:48.343725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8068b2-06b2-45a7-8147-ef1234dbf31f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.344022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8068b2-06b2-45a7-8147-ef1234dbf31f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.344305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8068b2-06b2-45a7-8147-ef1234dbf31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.360546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3f8068b2-06b2-45a7-8147-ef1234dbf31f None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:48.361751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3f8068b2-06b2-45a7-8147-ef1234dbf31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:48.372963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 39/77] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:48.380262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94189c8-7215-4c7a-ae77-7f2209c85c46 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:48.398770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:48.428072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 39/78] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:48.428916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8068b2-06b2-45a7-8147-ef1234dbf31f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:48.468652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:48.487931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94189c8-7215-4c7a-ae77-7f2209c85c46 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.521561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8de5e-b798-45e6-bb9c-b52d602072ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.522222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8de5e-b798-45e6-bb9c-b52d602072ce None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.522512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8de5e-b798-45e6-bb9c-b52d602072ce None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.522792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8de5e-b798-45e6-bb9c-b52d602072ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.525757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8068b2-06b2-45a7-8147-ef1234dbf31f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.558977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d11f1-db61-4146-a4ed-e519caa005b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.559869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d11f1-db61-4146-a4ed-e519caa005b5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.560157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d11f1-db61-4146-a4ed-e519caa005b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.560436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d11f1-db61-4146-a4ed-e519caa005b5 None admin] PATH_INFO: `/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.561625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d11f1-db61-4146-a4ed-e519caa005b5 None admin] RBAC: Authorizing `identity:get_user(user_id=7b7d0fd6414b417ab8bf3499e0f4961b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:48.563149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d11f1-db61-4146-a4ed-e519caa005b5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:48.564757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 40/79] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:48 2022] GET /identity/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:48.565511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d11f1-db61-4146-a4ed-e519caa005b5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:48.593571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8de5e-b798-45e6-bb9c-b52d602072ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:48.595069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8de5e-b798-45e6-bb9c-b52d602072ce None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:48.609294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:48.664425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d11f1-db61-4146-a4ed-e519caa005b5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.671746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:48.693999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 40/80] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 14:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 14:59:48.694850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8de5e-b798-45e6-bb9c-b52d602072ce None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:48.700261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568d885-7eee-497b-9186-e8f770ecf301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.701150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568d885-7eee-497b-9186-e8f770ecf301 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.701442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568d885-7eee-497b-9186-e8f770ecf301 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.701722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568d885-7eee-497b-9186-e8f770ecf301 None admin] PATH_INFO: `/v3/users/49df8493343b42b0bb70939a8aa8aee1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.703264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568d885-7eee-497b-9186-e8f770ecf301 None admin] RBAC: Authorizing `identity:get_user(user_id=49df8493343b42b0bb70939a8aa8aee1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:48.704734 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a568d885-7eee-497b-9186-e8f770ecf301 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:48.706356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 41/81] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:48 2022] GET /identity/v3/users/49df8493343b42b0bb70939a8aa8aee1 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:48.707138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568d885-7eee-497b-9186-e8f770ecf301 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:48.794840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8de5e-b798-45e6-bb9c-b52d602072ce None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.803986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568d885-7eee-497b-9186-e8f770ecf301 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.809007 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:48.829991 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973301d9-77dd-4bd0-8d8e-f6c91ce4c464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.830779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973301d9-77dd-4bd0-8d8e-f6c91ce4c464 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.831428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973301d9-77dd-4bd0-8d8e-f6c91ce4c464 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.831719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973301d9-77dd-4bd0-8d8e-f6c91ce4c464 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.832878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973301d9-77dd-4bd0-8d8e-f6c91ce4c464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:48.834325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973301d9-77dd-4bd0-8d8e-f6c91ce4c464 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:48.836019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 41/82] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:48.837361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1680847-04b8-4d9b-afd7-d86e11467c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.838161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1680847-04b8-4d9b-afd7-d86e11467c86 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.838454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1680847-04b8-4d9b-afd7-d86e11467c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.838746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1680847-04b8-4d9b-afd7-d86e11467c86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.841144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973301d9-77dd-4bd0-8d8e-f6c91ce4c464 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:48.849066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1680847-04b8-4d9b-afd7-d86e11467c86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f1680847-04b8-4d9b-afd7-d86e11467c86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:48.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:48.863777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:48.864535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 42/83] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:48.866468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1680847-04b8-4d9b-afd7-d86e11467c86 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:48.900827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:48.939133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973301d9-77dd-4bd0-8d8e-f6c91ce4c464 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.964982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1680847-04b8-4d9b-afd7-d86e11467c86 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:48.973742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6f1d-1465-4144-99c8-47d15d2db98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.974447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6f1d-1465-4144-99c8-47d15d2db98f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.974736 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6f1d-1465-4144-99c8-47d15d2db98f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:48.975012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e6f1d-1465-4144-99c8-47d15d2db98f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:48.998714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84255e83-4427-4c93-9fc5-7de022df634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:48.999658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84255e83-4427-4c93-9fc5-7de022df634d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:48.999947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84255e83-4427-4c93-9fc5-7de022df634d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:49.000226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84255e83-4427-4c93-9fc5-7de022df634d None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:49.001417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84255e83-4427-4c93-9fc5-7de022df634d None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:49.002866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84255e83-4427-4c93-9fc5-7de022df634d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:49.009300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 43/84] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:48 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:49.009300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84255e83-4427-4c93-9fc5-7de022df634d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:49.028760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:49.049609 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6f1d-1465-4144-99c8-47d15d2db98f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:49.051026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e6f1d-1465-4144-99c8-47d15d2db98f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:49.103636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84255e83-4427-4c93-9fc5-7de022df634d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:49.105228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:49.138494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa508b-567c-4104-95db-6568648aaed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:49.139176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa508b-567c-4104-95db-6568648aaed9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:49.139471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa508b-567c-4104-95db-6568648aaed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:49.139760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa508b-567c-4104-95db-6568648aaed9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:49.149620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 42/85] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 14:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 14:59:49.150430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6f1d-1465-4144-99c8-47d15d2db98f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:49.213763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa508b-567c-4104-95db-6568648aaed9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:49.215298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa508b-567c-4104-95db-6568648aaed9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:49.229091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:49.247232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e6f1d-1465-4144-99c8-47d15d2db98f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:49.282031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d45e45-a318-4b13-8de6-d35fce68f35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:49.282703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d45e45-a318-4b13-8de6-d35fce68f35d None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:49.282990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d45e45-a318-4b13-8de6-d35fce68f35d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:49.283299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d45e45-a318-4b13-8de6-d35fce68f35d None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:49.284027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d45e45-a318-4b13-8de6-d35fce68f35d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:49.285488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d45e45-a318-4b13-8de6-d35fce68f35d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:49.304364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 43/86] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 14:59:49 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 14:59:49.305103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d45e45-a318-4b13-8de6-d35fce68f35d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:49.316386 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 44/87] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 14:59:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 14:59:49.317543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa508b-567c-4104-95db-6568648aaed9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:49.323841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:49.431765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d45e45-a318-4b13-8de6-d35fce68f35d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:49.440944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa508b-567c-4104-95db-6568648aaed9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:49.475556 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47dee-ca25-491c-81b2-272a1fe2c404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:49.476706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47dee-ca25-491c-81b2-272a1fe2c404 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:49.476881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47dee-ca25-491c-81b2-272a1fe2c404 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:49.477596 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a47dee-ca25-491c-81b2-272a1fe2c404 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:49.477897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47dee-ca25-491c-81b2-272a1fe2c404 None admin] RBAC: Adding query filter params (name=member) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:49.478212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47dee-ca25-491c-81b2-272a1fe2c404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:49.479689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a47dee-ca25-491c-81b2-272a1fe2c404 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:49.484551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a888a0f-155c-4508-ac04-8d19ba5954ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:49.484551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a888a0f-155c-4508-ac04-8d19ba5954ee None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:49.484740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a888a0f-155c-4508-ac04-8d19ba5954ee None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:49.484965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a888a0f-155c-4508-ac04-8d19ba5954ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:49.485685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:49.485790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a888a0f-155c-4508-ac04-8d19ba5954ee None admin] RBAC: Adding query filter params (role.id=6e32448ba01f41bcafb4cb0a839691fe, scope.project.id=a2c6e66812894558944eb8f6f156896d, user.id=49df8493343b42b0bb70939a8aa8aee1) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:49.486618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a888a0f-155c-4508-ac04-8d19ba5954ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:49.489526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a888a0f-155c-4508-ac04-8d19ba5954ee None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:49.489668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 44/88] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 14:59:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:49.490412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47dee-ca25-491c-81b2-272a1fe2c404 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:49.498008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 45/89] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 14:59:49 2022] GET /identity/v3/role_assignments?user.id=49df8493343b42b0bb70939a8aa8aee1&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:49.499109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a888a0f-155c-4508-ac04-8d19ba5954ee None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:49.590019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:49.641826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a888a0f-155c-4508-ac04-8d19ba5954ee None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:49.642012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a47dee-ca25-491c-81b2-272a1fe2c404 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:49.685786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548b921-e373-48fd-9eb5-80786a97ed7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:49.686753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548b921-e373-48fd-9eb5-80786a97ed7f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:49.687072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548b921-e373-48fd-9eb5-80786a97ed7f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:49.687414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4548b921-e373-48fd-9eb5-80786a97ed7f None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:49.688185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566072ba-e607-4d6c-8ba9-1098db2ac45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:49.688737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548b921-e373-48fd-9eb5-80786a97ed7f None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:49.689069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566072ba-e607-4d6c-8ba9-1098db2ac45c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:49.689362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566072ba-e607-4d6c-8ba9-1098db2ac45c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:49.689645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566072ba-e607-4d6c-8ba9-1098db2ac45c None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:49.690349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4548b921-e373-48fd-9eb5-80786a97ed7f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:49.690867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566072ba-e607-4d6c-8ba9-1098db2ac45c None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:49.692370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 46/90] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:49 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:49.692475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566072ba-e607-4d6c-8ba9-1098db2ac45c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:49.692915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548b921-e373-48fd-9eb5-80786a97ed7f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:49.694395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 45/91] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:49 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:49.699459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566072ba-e607-4d6c-8ba9-1098db2ac45c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:49.770459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:49.806525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566072ba-e607-4d6c-8ba9-1098db2ac45c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:49.812222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4548b921-e373-48fd-9eb5-80786a97ed7f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:49.841650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0203098-99d1-4afd-8eff-b5cd3d2231fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:49.842233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0203098-99d1-4afd-8eff-b5cd3d2231fb None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:49.842524 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0203098-99d1-4afd-8eff-b5cd3d2231fb None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:49.842836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0203098-99d1-4afd-8eff-b5cd3d2231fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:49.843757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0203098-99d1-4afd-8eff-b5cd3d2231fb None admin] RBAC: Adding query filter params (role.id=4301bced0259446cafb79118a53a015e, scope.project.id=a2c6e66812894558944eb8f6f156896d, user.id=49df8493343b42b0bb70939a8aa8aee1) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:49.844081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0203098-99d1-4afd-8eff-b5cd3d2231fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:49.845548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0203098-99d1-4afd-8eff-b5cd3d2231fb None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:49.850017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda5f93-190a-4e45-9f89-86ece6fa6190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:49.850904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda5f93-190a-4e45-9f89-86ece6fa6190 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:49.851246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda5f93-190a-4e45-9f89-86ece6fa6190 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:49.851530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda5f93-190a-4e45-9f89-86ece6fa6190 None admin] PATH_INFO: `/v3/users/49df8493343b42b0bb70939a8aa8aee1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:49.851819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 46/92] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 14:59:49 2022] GET /identity/v3/role_assignments?user.id=49df8493343b42b0bb70939a8aa8aee1&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=4301bced0259446cafb79118a53a015e => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:49.852778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0203098-99d1-4afd-8eff-b5cd3d2231fb None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:49.853077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda5f93-190a-4e45-9f89-86ece6fa6190 None admin] RBAC: Authorizing `identity:get_user(user_id=49df8493343b42b0bb70939a8aa8aee1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:49.854569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda5f93-190a-4e45-9f89-86ece6fa6190 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:49.856778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 47/93] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:49 2022] GET /identity/v3/users/49df8493343b42b0bb70939a8aa8aee1 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:49.863370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda5f93-190a-4e45-9f89-86ece6fa6190 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:49.866429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:49.961866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0203098-99d1-4afd-8eff-b5cd3d2231fb None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:49.984622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda5f93-190a-4e45-9f89-86ece6fa6190 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:49.994352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:50.011863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b4746-1234-405c-80ba-25cf7a727eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:50.012649 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b4746-1234-405c-80ba-25cf7a727eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:50.012849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b4746-1234-405c-80ba-25cf7a727eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:50.013187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b4746-1234-405c-80ba-25cf7a727eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:50.014016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b4746-1234-405c-80ba-25cf7a727eb2 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=a2c6e66812894558944eb8f6f156896d, user.id=7b7d0fd6414b417ab8bf3499e0f4961b) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:50.014338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b4746-1234-405c-80ba-25cf7a727eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:50.015984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b4746-1234-405c-80ba-25cf7a727eb2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:50.022067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 47/94] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 14:59:49 2022] GET /identity/v3/role_assignments?user.id=7b7d0fd6414b417ab8bf3499e0f4961b&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:50.028305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4263ac-eea7-4944-9105-8a7fac8fb64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:50.029242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4263ac-eea7-4944-9105-8a7fac8fb64a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:50.029684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4263ac-eea7-4944-9105-8a7fac8fb64a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:50.030115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4263ac-eea7-4944-9105-8a7fac8fb64a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:50.112488 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4263ac-eea7-4944-9105-8a7fac8fb64a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:50.114370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4263ac-eea7-4944-9105-8a7fac8fb64a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:50.116871 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:50.229705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 48/95] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 14:59:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 14:59:50.235585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b4746-1234-405c-80ba-25cf7a727eb2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:50.357500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b4746-1234-405c-80ba-25cf7a727eb2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:50.393297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71415fde-e888-4962-9dcd-666e0db70188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:50.394174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71415fde-e888-4962-9dcd-666e0db70188 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:50.394463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71415fde-e888-4962-9dcd-666e0db70188 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:50.394744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71415fde-e888-4962-9dcd-666e0db70188 None admin] PATH_INFO: `/v3/projects/e59fd922239b4521b0c65581d18ac618` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:50.395980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71415fde-e888-4962-9dcd-666e0db70188 None admin] RBAC: Authorizing `identity:get_project(project_id=e59fd922239b4521b0c65581d18ac618)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:50.397464 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71415fde-e888-4962-9dcd-666e0db70188 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:50.399509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 48/96] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:50 2022] GET /identity/v3/projects/e59fd922239b4521b0c65581d18ac618 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:50.408359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4263ac-eea7-4944-9105-8a7fac8fb64a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:50.573795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4263ac-eea7-4944-9105-8a7fac8fb64a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:50.575968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:50.616535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f675cac-f313-48f0-a039-52e2c60cff6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:50.617431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f675cac-f313-48f0-a039-52e2c60cff6f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:50.617870 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f675cac-f313-48f0-a039-52e2c60cff6f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:50.618300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f675cac-f313-48f0-a039-52e2c60cff6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:50.619359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f675cac-f313-48f0-a039-52e2c60cff6f None admin] RBAC: Adding query filter params (role.id=6e32448ba01f41bcafb4cb0a839691fe, scope.project.id=e59fd922239b4521b0c65581d18ac618, user.id=49df8493343b42b0bb70939a8aa8aee1) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:50.619841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f675cac-f313-48f0-a039-52e2c60cff6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:50.621424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f675cac-f313-48f0-a039-52e2c60cff6f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:50.627497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 49/97] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 14:59:50 2022] GET /identity/v3/role_assignments?user.id=49df8493343b42b0bb70939a8aa8aee1&scope.project.id=e59fd922239b4521b0c65581d18ac618&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:52.221228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32e8a0-9ae3-4fe7-922c-c2cd35c1a003 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:52.221759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32e8a0-9ae3-4fe7-922c-c2cd35c1a003 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:52.221835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32e8a0-9ae3-4fe7-922c-c2cd35c1a003 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:52.223478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 49/98] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:52.228862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e85592-ff5d-4a61-a562-8914a735413c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:52.229320 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e85592-ff5d-4a61-a562-8914a735413c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:52.229504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e85592-ff5d-4a61-a562-8914a735413c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:52.251654 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-34e85592-ff5d-4a61-a562-8914a735413c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:52.252891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-34e85592-ff5d-4a61-a562-8914a735413c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:52.336389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 50/99] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:52.370721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32e8a0-9ae3-4fe7-922c-c2cd35c1a003 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:52.456486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:52.515499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c32e8a0-9ae3-4fe7-922c-c2cd35c1a003 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:52.563894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d5bfc2-ff81-41ae-95da-fe7f8b0725f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:52.564706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5bfc2-ff81-41ae-95da-fe7f8b0725f0 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:52.564998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5bfc2-ff81-41ae-95da-fe7f8b0725f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:52.565324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d5bfc2-ff81-41ae-95da-fe7f8b0725f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:52.581345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d5bfc2-ff81-41ae-95da-fe7f8b0725f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-68d5bfc2-ff81-41ae-95da-fe7f8b0725f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:52.589207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:52.590043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 14:59:52.590808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 50/100] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:52.597139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e85592-ff5d-4a61-a562-8914a735413c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:52.615706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb16175-d16c-43c3-99d5-840b4e72b2d0 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:52.615999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb16175-d16c-43c3-99d5-840b4e72b2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:52.616290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb16175-d16c-43c3-99d5-840b4e72b2d0 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:52.617812 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 51/101] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:52.622619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86498a4c-581b-48f6-944a-c81e564f846c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:52.622919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86498a4c-581b-48f6-944a-c81e564f846c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:52.623235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86498a4c-581b-48f6-944a-c81e564f846c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:52.658359 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-86498a4c-581b-48f6-944a-c81e564f846c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:52.659603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-86498a4c-581b-48f6-944a-c81e564f846c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:52.775799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:52.779958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e85592-ff5d-4a61-a562-8914a735413c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:52.802192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 52/102] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:52.803616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590185fb-18d0-4fec-a562-1869283330a3 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:52.803870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590185fb-18d0-4fec-a562-1869283330a3 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:52.804153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590185fb-18d0-4fec-a562-1869283330a3 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:52.805792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 53/103] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:52.807073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a8dea-c73b-4471-8146-33ce8beb4e59 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:52.807449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a8dea-c73b-4471-8146-33ce8beb4e59 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:52.807651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a8dea-c73b-4471-8146-33ce8beb4e59 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:52.809086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 54/104] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:52.814585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd8eb6-6c09-42f8-81f2-6ffef40b55a4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:52.814864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd8eb6-6c09-42f8-81f2-6ffef40b55a4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:52.815200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd8eb6-6c09-42f8-81f2-6ffef40b55a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:52.819554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b33f0f-cec1-4920-aaee-9aa950c9d455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:52.820484 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b33f0f-cec1-4920-aaee-9aa950c9d455 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:52.820967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b33f0f-cec1-4920-aaee-9aa950c9d455 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:52.821453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b33f0f-cec1-4920-aaee-9aa950c9d455 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:52.822406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b33f0f-cec1-4920-aaee-9aa950c9d455 None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:52.822912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b33f0f-cec1-4920-aaee-9aa950c9d455 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:52.824539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b33f0f-cec1-4920-aaee-9aa950c9d455 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:52.832241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-93cd8eb6-6c09-42f8-81f2-6ffef40b55a4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:52.833527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-93cd8eb6-6c09-42f8-81f2-6ffef40b55a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:52.834362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 51/105] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 14:59:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:52.835825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a5da7-d931-4876-a2e4-8dcd90c2f739 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:52.836345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a5da7-d931-4876-a2e4-8dcd90c2f739 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:52.836831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07a5da7-d931-4876-a2e4-8dcd90c2f739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:52.853491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e07a5da7-d931-4876-a2e4-8dcd90c2f739 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:52.854990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e07a5da7-d931-4876-a2e4-8dcd90c2f739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:52.898988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:52.918767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 55/106] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:52.918767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd8eb6-6c09-42f8-81f2-6ffef40b55a4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:52.955697 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 52/107] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:52.956856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a5da7-d931-4876-a2e4-8dcd90c2f739 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:53.044562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd8eb6-6c09-42f8-81f2-6ffef40b55a4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.051232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:53.064539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07a5da7-d931-4876-a2e4-8dcd90c2f739 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.083466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a3c4c-c829-4f43-a564-e03c81dff33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:53.084584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a3c4c-c829-4f43-a564-e03c81dff33b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.085048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a3c4c-c829-4f43-a564-e03c81dff33b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.085474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a3c4c-c829-4f43-a564-e03c81dff33b None admin] PATH_INFO: `/v3/users/49df8493343b42b0bb70939a8aa8aee1` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.086943 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a3c4c-c829-4f43-a564-e03c81dff33b None admin] RBAC: Authorizing `identity:get_user(user_id=49df8493343b42b0bb70939a8aa8aee1)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:53.088583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a3c4c-c829-4f43-a564-e03c81dff33b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:53.090608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 56/108] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:52 2022] GET /identity/v3/users/49df8493343b42b0bb70939a8aa8aee1 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:53.091562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a3c4c-c829-4f43-a564-e03c81dff33b None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:53.101790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ade90a-fdf6-4232-82d9-15d72513fbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:53.102747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ade90a-fdf6-4232-82d9-15d72513fbfa None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.103164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ade90a-fdf6-4232-82d9-15d72513fbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.103563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ade90a-fdf6-4232-82d9-15d72513fbfa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.114470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ade90a-fdf6-4232-82d9-15d72513fbfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-81ade90a-fdf6-4232-82d9-15d72513fbfa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:53.122517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:53.123628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 14:59:53.125133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:53.126886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 53/109] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:53.127857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ade90a-fdf6-4232-82d9-15d72513fbfa None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:53.152423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:53.203934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a3c4c-c829-4f43-a564-e03c81dff33b None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.241009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12c96a-3c35-4780-b511-1b3291ab1669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:53.241961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12c96a-3c35-4780-b511-1b3291ab1669 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12c96a-3c35-4780-b511-1b3291ab1669 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.242737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12c96a-3c35-4780-b511-1b3291ab1669 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.244339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12c96a-3c35-4780-b511-1b3291ab1669 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:53.245899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12c96a-3c35-4780-b511-1b3291ab1669 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:53.246972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ade90a-fdf6-4232-82d9-15d72513fbfa None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.247768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 57/110] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:53.248626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12c96a-3c35-4780-b511-1b3291ab1669 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:53.283403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b4bbe-7d84-4901-b8be-352af610ca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:53.284481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b4bbe-7d84-4901-b8be-352af610ca3a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.285009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b4bbe-7d84-4901-b8be-352af610ca3a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.285428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b4bbe-7d84-4901-b8be-352af610ca3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.297451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b4bbe-7d84-4901-b8be-352af610ca3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-865b4bbe-7d84-4901-b8be-352af610ca3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:53.306347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:53.307253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 14:59:53.308018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:53.310489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 54/111] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 14:59:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:53.316220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b4bbe-7d84-4901-b8be-352af610ca3a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:53.328890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:53.358406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12c96a-3c35-4780-b511-1b3291ab1669 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.380368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:53.396142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbe01d-d298-4fbd-8070-396bd19f9c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:53.397121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbe01d-d298-4fbd-8070-396bd19f9c33 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.397414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbe01d-d298-4fbd-8070-396bd19f9c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.397705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbe01d-d298-4fbd-8070-396bd19f9c33 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.399014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbe01d-d298-4fbd-8070-396bd19f9c33 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:53.400645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbe01d-d298-4fbd-8070-396bd19f9c33 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:53.402703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 58/112] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:53 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:53.403638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbe01d-d298-4fbd-8070-396bd19f9c33 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:53.436285 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b4bbe-7d84-4901-b8be-352af610ca3a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.470217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635b8da-cca6-4872-a52e-7c8acaa934bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:53.470897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635b8da-cca6-4872-a52e-7c8acaa934bd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.471229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635b8da-cca6-4872-a52e-7c8acaa934bd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.471519 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635b8da-cca6-4872-a52e-7c8acaa934bd None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.472281 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635b8da-cca6-4872-a52e-7c8acaa934bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:53.472603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635b8da-cca6-4872-a52e-7c8acaa934bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:53.474015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5635b8da-cca6-4872-a52e-7c8acaa934bd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:53.481896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 55/113] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 14:59:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:53.482678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635b8da-cca6-4872-a52e-7c8acaa934bd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:53.504383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbe01d-d298-4fbd-8070-396bd19f9c33 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.537721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eb231-79f3-416d-90d4-12bfb99ca6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:53.538379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eb231-79f3-416d-90d4-12bfb99ca6aa None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.538663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eb231-79f3-416d-90d4-12bfb99ca6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.538945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eb231-79f3-416d-90d4-12bfb99ca6aa None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.539689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eb231-79f3-416d-90d4-12bfb99ca6aa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:53.541130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eb231-79f3-416d-90d4-12bfb99ca6aa None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:53.551160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:53.565062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 59/114] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 14:59:53 2022] POST /identity/v3/projects => generated 302 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 14:59:53.566477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33173c7-e642-4c5c-bd1b-06895e2369f4 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.566775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33173c7-e642-4c5c-bd1b-06895e2369f4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.567077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33173c7-e642-4c5c-bd1b-06895e2369f4 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.571452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 60/115] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:53.571452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33173c7-e642-4c5c-bd1b-06895e2369f4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:53.587637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635b8da-cca6-4872-a52e-7c8acaa934bd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.625237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf55668-0703-4acd-9f5a-b3d749722b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:53.625929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf55668-0703-4acd-9f5a-b3d749722b2c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.626215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf55668-0703-4acd-9f5a-b3d749722b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.626494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf55668-0703-4acd-9f5a-b3d749722b2c None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.627293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf55668-0703-4acd-9f5a-b3d749722b2c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:53.627610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf55668-0703-4acd-9f5a-b3d749722b2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:53.629069 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf55668-0703-4acd-9f5a-b3d749722b2c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:53.631303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:53.640867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 56/116] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 14:59:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:53.642335 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729b27-7d09-4468-ab09-81ce232e852f None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.642637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729b27-7d09-4468-ab09-81ce232e852f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.642920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc729b27-7d09-4468-ab09-81ce232e852f None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.644317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 57/117] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:53.647255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729b27-7d09-4468-ab09-81ce232e852f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:53.709576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33173c7-e642-4c5c-bd1b-06895e2369f4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.752018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4118e6-f0e2-4c00-b646-e1db9a286c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:53.752961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4118e6-f0e2-4c00-b646-e1db9a286c9d None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.753256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4118e6-f0e2-4c00-b646-e1db9a286c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.753540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4118e6-f0e2-4c00-b646-e1db9a286c9d None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d/users/49df8493343b42b0bb70939a8aa8aee1/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.756537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4118e6-f0e2-4c00-b646-e1db9a286c9d None admin] RBAC: Authorizing `identity:create_grant(project_id=a2c6e66812894558944eb8f6f156896d, user_id=49df8493343b42b0bb70939a8aa8aee1, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:53.758079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4118e6-f0e2-4c00-b646-e1db9a286c9d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:53.771565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc729b27-7d09-4468-ab09-81ce232e852f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.776639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 61/118] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 14:59:53 2022] PUT /identity/v3/projects/a2c6e66812894558944eb8f6f156896d/users/49df8493343b42b0bb70939a8aa8aee1/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 14:59:53.778103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252012e-2993-4c8e-8b18-2ef8280812c4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.778466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252012e-2993-4c8e-8b18-2ef8280812c4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.778809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1252012e-2993-4c8e-8b18-2ef8280812c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.801474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-1252012e-2993-4c8e-8b18-2ef8280812c4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:53.802846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-1252012e-2993-4c8e-8b18-2ef8280812c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:53.813649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:53.820042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08a1bf-4a3a-4d11-a1f3-b43927a396d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:53.821231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08a1bf-4a3a-4d11-a1f3-b43927a396d2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.821702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08a1bf-4a3a-4d11-a1f3-b43927a396d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.822203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08a1bf-4a3a-4d11-a1f3-b43927a396d2 None admin] PATH_INFO: `/v3/users/013c7ef73b0e4bc0b335f79c4f2a2ffa` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.838655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08a1bf-4a3a-4d11-a1f3-b43927a396d2 None admin] RBAC: Authorizing `identity:get_user(user_id=013c7ef73b0e4bc0b335f79c4f2a2ffa)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:53.840500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08a1bf-4a3a-4d11-a1f3-b43927a396d2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:53.842468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 58/119] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:53 2022] GET /identity/v3/users/013c7ef73b0e4bc0b335f79c4f2a2ffa => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:53.847208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08a1bf-4a3a-4d11-a1f3-b43927a396d2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:53.905745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 62/120] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:53.915600 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcae55-e120-41c7-b861-abd95d544839 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:53.916263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcae55-e120-41c7-b861-abd95d544839 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:53.916753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcae55-e120-41c7-b861-abd95d544839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:53.948156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-20fcae55-e120-41c7-b861-abd95d544839 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:53.949709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-20fcae55-e120-41c7-b861-abd95d544839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:53.966539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08a1bf-4a3a-4d11-a1f3-b43927a396d2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:53.977779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:54.006815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffda44-faa6-4ea8-a3a9-53e57c2e586f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:54.007918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffda44-faa6-4ea8-a3a9-53e57c2e586f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:54.008316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffda44-faa6-4ea8-a3a9-53e57c2e586f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:54.008744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ffda44-faa6-4ea8-a3a9-53e57c2e586f None admin] PATH_INFO: `/v3/users/49df8493343b42b0bb70939a8aa8aee1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:54.010107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffda44-faa6-4ea8-a3a9-53e57c2e586f None admin] RBAC: Authorizing `identity:get_user(user_id=49df8493343b42b0bb70939a8aa8aee1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:54.011787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ffda44-faa6-4ea8-a3a9-53e57c2e586f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:54.013778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 59/121] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:53 2022] GET /identity/v3/users/49df8493343b42b0bb70939a8aa8aee1 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:54.015007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffda44-faa6-4ea8-a3a9-53e57c2e586f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:54.039583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 63/122] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:54.040743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcae55-e120-41c7-b861-abd95d544839 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:54.093437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:54.141597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ffda44-faa6-4ea8-a3a9-53e57c2e586f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:54.179208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e00b3-50f5-425f-a3e2-2dc0897b20c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:54.179549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcae55-e120-41c7-b861-abd95d544839 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:54.180108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e00b3-50f5-425f-a3e2-2dc0897b20c2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:54.180501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e00b3-50f5-425f-a3e2-2dc0897b20c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:54.180890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e00b3-50f5-425f-a3e2-2dc0897b20c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:54.226016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be1555-d008-4722-90b5-59d8a0496add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:54.227081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be1555-d008-4722-90b5-59d8a0496add None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:54.227513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be1555-d008-4722-90b5-59d8a0496add None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:54.227930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72be1555-d008-4722-90b5-59d8a0496add None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:54.243922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72be1555-d008-4722-90b5-59d8a0496add None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-72be1555-d008-4722-90b5-59d8a0496add None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:54.255751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:54.256809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 14:59:54.257755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 14:59:54.260546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 64/123] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 14:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:54.261530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be1555-d008-4722-90b5-59d8a0496add None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:54.274479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e00b3-50f5-425f-a3e2-2dc0897b20c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:54.276966 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048e00b3-50f5-425f-a3e2-2dc0897b20c2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:54.288601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:54.351064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:54.378495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72be1555-d008-4722-90b5-59d8a0496add None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:54.408884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 60/124] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 14:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 14:59:54.410224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e00b3-50f5-425f-a3e2-2dc0897b20c2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:54.419740 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788371b-03ca-43ed-aa1b-bf26b974aadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:54.420607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788371b-03ca-43ed-aa1b-bf26b974aadf None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:54.420932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788371b-03ca-43ed-aa1b-bf26b974aadf None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:54.421497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788371b-03ca-43ed-aa1b-bf26b974aadf None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:54.422393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788371b-03ca-43ed-aa1b-bf26b974aadf None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:54.423885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788371b-03ca-43ed-aa1b-bf26b974aadf None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:54.425553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 65/125] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:54 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:54.426378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788371b-03ca-43ed-aa1b-bf26b974aadf None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:54.524689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:54.526166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e00b3-50f5-425f-a3e2-2dc0897b20c2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:54.538805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788371b-03ca-43ed-aa1b-bf26b974aadf None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:54.572742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1b3d7-8a6c-4b7f-8333-2c7fb32994d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:54.573879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1b3d7-8a6c-4b7f-8333-2c7fb32994d8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:54.574365 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1b3d7-8a6c-4b7f-8333-2c7fb32994d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:54.574841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1b3d7-8a6c-4b7f-8333-2c7fb32994d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:54.574963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16052e36-c3de-4433-b0ac-83f1d975c3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:54.575614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16052e36-c3de-4433-b0ac-83f1d975c3bd None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:54.575997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16052e36-c3de-4433-b0ac-83f1d975c3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:54.576423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16052e36-c3de-4433-b0ac-83f1d975c3bd None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:54.578057 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16052e36-c3de-4433-b0ac-83f1d975c3bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:54.578372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16052e36-c3de-4433-b0ac-83f1d975c3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:54.579899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16052e36-c3de-4433-b0ac-83f1d975c3bd None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:54.588660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 66/126] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:54.589544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16052e36-c3de-4433-b0ac-83f1d975c3bd None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:54.589963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1b3d7-8a6c-4b7f-8333-2c7fb32994d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e5b1b3d7-8a6c-4b7f-8333-2c7fb32994d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:54.599804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:54.600693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 14:59:54.601508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:54.604280 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 61/127] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:54.605349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1b3d7-8a6c-4b7f-8333-2c7fb32994d8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:54.647422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:54.696673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16052e36-c3de-4433-b0ac-83f1d975c3bd None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:54.735172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fdcd9-1ead-455a-9451-fc2587f65604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:54.736289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fdcd9-1ead-455a-9451-fc2587f65604 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:54.736679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fdcd9-1ead-455a-9451-fc2587f65604 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:54.737085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0fdcd9-1ead-455a-9451-fc2587f65604 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:54.737837 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1b3d7-8a6c-4b7f-8333-2c7fb32994d8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:54.738679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fdcd9-1ead-455a-9451-fc2587f65604 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:54.740377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0fdcd9-1ead-455a-9451-fc2587f65604 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:54.747131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 67/128] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:54 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:54.748213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fdcd9-1ead-455a-9451-fc2587f65604 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:54.768972 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:54.782938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68556070-a891-4019-aec7-9d8b162d61b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:54.784688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68556070-a891-4019-aec7-9d8b162d61b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:54.785459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68556070-a891-4019-aec7-9d8b162d61b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:54.785888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68556070-a891-4019-aec7-9d8b162d61b5 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d/users/49df8493343b42b0bb70939a8aa8aee1/roles/4301bced0259446cafb79118a53a015e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:54.789483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68556070-a891-4019-aec7-9d8b162d61b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2c6e66812894558944eb8f6f156896d, user_id=49df8493343b42b0bb70939a8aa8aee1, role_id=4301bced0259446cafb79118a53a015e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:54.791167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68556070-a891-4019-aec7-9d8b162d61b5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:54.813615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 62/129] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 14:59:54 2022] PUT /identity/v3/projects/a2c6e66812894558944eb8f6f156896d/users/49df8493343b42b0bb70939a8aa8aee1/roles/4301bced0259446cafb79118a53a015e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 14:59:54.815244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68556070-a891-4019-aec7-9d8b162d61b5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:54.896862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0fdcd9-1ead-455a-9451-fc2587f65604 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:54.971353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:54.974282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc913a-ca11-4d87-a733-471eb3571b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:54.975689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc913a-ca11-4d87-a733-471eb3571b5c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:54.977691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc913a-ca11-4d87-a733-471eb3571b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:54.979913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bc913a-ca11-4d87-a733-471eb3571b5c None admin] PATH_INFO: `/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:54.983745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc913a-ca11-4d87-a733-471eb3571b5c None admin] RBAC: Authorizing `identity:get_user(user_id=7b7d0fd6414b417ab8bf3499e0f4961b)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:54.985221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bc913a-ca11-4d87-a733-471eb3571b5c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:54.991153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 68/130] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:54 2022] GET /identity/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b => generated 247 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:54.992300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc913a-ca11-4d87-a733-471eb3571b5c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:55.004103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68556070-a891-4019-aec7-9d8b162d61b5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:55.053439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96de945-3a26-4b45-8186-72149e6834af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:55.054491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96de945-3a26-4b45-8186-72149e6834af None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:55.055359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96de945-3a26-4b45-8186-72149e6834af None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:55.055865 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96de945-3a26-4b45-8186-72149e6834af None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:55.056821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96de945-3a26-4b45-8186-72149e6834af None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:55.057330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96de945-3a26-4b45-8186-72149e6834af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:55.062316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96de945-3a26-4b45-8186-72149e6834af None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:55.074532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 63/131] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 14:59:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:55.075915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96de945-3a26-4b45-8186-72149e6834af None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:55.097851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:55.130505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bc913a-ca11-4d87-a733-471eb3571b5c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:55.196902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96de945-3a26-4b45-8186-72149e6834af None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:55.212910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486dc04-3f3f-4a9f-878d-f083681e952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:55.213587 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486dc04-3f3f-4a9f-878d-f083681e952b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:55.213883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486dc04-3f3f-4a9f-878d-f083681e952b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:55.214177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486dc04-3f3f-4a9f-878d-f083681e952b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:55.215095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486dc04-3f3f-4a9f-878d-f083681e952b None admin] RBAC: Adding query filter params (role.id=dd4438dfc8ea468db528cc4cda58f3b8, scope.project.id=a2c6e66812894558944eb8f6f156896d, user.id=013c7ef73b0e4bc0b335f79c4f2a2ffa) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:55.215425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486dc04-3f3f-4a9f-878d-f083681e952b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:55.217177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486dc04-3f3f-4a9f-878d-f083681e952b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:55.234330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 69/132] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 14:59:54 2022] GET /identity/v3/role_assignments?user.id=013c7ef73b0e4bc0b335f79c4f2a2ffa&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=dd4438dfc8ea468db528cc4cda58f3b8 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:55.235439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486dc04-3f3f-4a9f-878d-f083681e952b None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:55.236836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096334-492c-4e69-b1bc-ebced6395002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:55.237973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096334-492c-4e69-b1bc-ebced6395002 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:55.240290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096334-492c-4e69-b1bc-ebced6395002 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:55.240819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096334-492c-4e69-b1bc-ebced6395002 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:55.243603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d096334-492c-4e69-b1bc-ebced6395002 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:55.251826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d096334-492c-4e69-b1bc-ebced6395002 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:55.254127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 64/133] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:55 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:55.259567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096334-492c-4e69-b1bc-ebced6395002 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:55.279379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:55.401435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486dc04-3f3f-4a9f-878d-f083681e952b None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:55.441644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35346eba-695d-49b5-8bf2-77df637ebc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:55.442910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35346eba-695d-49b5-8bf2-77df637ebc25 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:55.443463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35346eba-695d-49b5-8bf2-77df637ebc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:55.443968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35346eba-695d-49b5-8bf2-77df637ebc25 None admin] PATH_INFO: `/v3/users/49df8493343b42b0bb70939a8aa8aee1` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:55.445897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35346eba-695d-49b5-8bf2-77df637ebc25 None admin] RBAC: Authorizing `identity:get_user(user_id=49df8493343b42b0bb70939a8aa8aee1)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:55.447687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35346eba-695d-49b5-8bf2-77df637ebc25 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:55.450891 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 70/134] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:55 2022] GET /identity/v3/users/49df8493343b42b0bb70939a8aa8aee1 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:55.460815 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35346eba-695d-49b5-8bf2-77df637ebc25 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:55.500221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:55.538572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096334-492c-4e69-b1bc-ebced6395002 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:55.629706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35346eba-695d-49b5-8bf2-77df637ebc25 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:55.666458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9054133-8d41-4294-802b-32eaf54c529d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:55.667745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9054133-8d41-4294-802b-32eaf54c529d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:55.668247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9054133-8d41-4294-802b-32eaf54c529d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:55.668724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9054133-8d41-4294-802b-32eaf54c529d None admin] PATH_INFO: `/v3/projects/e59fd922239b4521b0c65581d18ac618` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:55.670119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9054133-8d41-4294-802b-32eaf54c529d None admin] RBAC: Authorizing `identity:get_project(project_id=e59fd922239b4521b0c65581d18ac618)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:55.672369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9054133-8d41-4294-802b-32eaf54c529d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:55.674320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 71/135] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:55 2022] GET /identity/v3/projects/e59fd922239b4521b0c65581d18ac618 => generated 312 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:55.679819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e06634f-a565-4030-bef1-d04601ae2af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:55.680397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9054133-8d41-4294-802b-32eaf54c529d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:55.680744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06634f-a565-4030-bef1-d04601ae2af3 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:55.681095 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06634f-a565-4030-bef1-d04601ae2af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:55.687064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e06634f-a565-4030-bef1-d04601ae2af3 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d/users/7b7d0fd6414b417ab8bf3499e0f4961b/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:55.687924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06634f-a565-4030-bef1-d04601ae2af3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2c6e66812894558944eb8f6f156896d, user_id=7b7d0fd6414b417ab8bf3499e0f4961b, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:55.689465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e06634f-a565-4030-bef1-d04601ae2af3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:55.717205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 65/136] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 14:59:55 2022] PUT /identity/v3/projects/a2c6e66812894558944eb8f6f156896d/users/7b7d0fd6414b417ab8bf3499e0f4961b/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 14:59:55.828022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:55.848225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9054133-8d41-4294-802b-32eaf54c529d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:55.880836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711de906-0527-4787-ab8d-2eeb9fe39901 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:55.881210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711de906-0527-4787-ab8d-2eeb9fe39901 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:55.881505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711de906-0527-4787-ab8d-2eeb9fe39901 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:55.883034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 66/137] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:55.888819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f21f20-8258-4b37-948a-ac46633d6543 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:55.889116 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f21f20-8258-4b37-948a-ac46633d6543 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:55.889449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f21f20-8258-4b37-948a-ac46633d6543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:55.911942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c96b5-ace1-4262-90eb-36a05b83b0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:55.912865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c96b5-ace1-4262-90eb-36a05b83b0c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:55.913150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c96b5-ace1-4262-90eb-36a05b83b0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:55.913428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84c96b5-ace1-4262-90eb-36a05b83b0c7 None admin] PATH_INFO: `/v3/projects/e59fd922239b4521b0c65581d18ac618/users/49df8493343b42b0bb70939a8aa8aee1/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:55.920144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c96b5-ace1-4262-90eb-36a05b83b0c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e59fd922239b4521b0c65581d18ac618, user_id=49df8493343b42b0bb70939a8aa8aee1, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:55.921769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84c96b5-ace1-4262-90eb-36a05b83b0c7 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:55.931464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f3f21f20-8258-4b37-948a-ac46633d6543 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:55.933012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f3f21f20-8258-4b37-948a-ac46633d6543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:55.954458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 72/138] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 14:59:55 2022] PUT /identity/v3/projects/e59fd922239b4521b0c65581d18ac618/users/49df8493343b42b0bb70939a8aa8aee1/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 14:59:56.016885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:56.025473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 67/139] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:56.049252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c96b5-ace1-4262-90eb-36a05b83b0c7 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:56.162383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84c96b5-ace1-4262-90eb-36a05b83b0c7 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:56.210834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5d461-ccc6-429b-9031-5361d8a54248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:56.211907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d461-ccc6-429b-9031-5361d8a54248 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:56.212333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d461-ccc6-429b-9031-5361d8a54248 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:56.212770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5d461-ccc6-429b-9031-5361d8a54248 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:56.214091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d461-ccc6-429b-9031-5361d8a54248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:56.215731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5d461-ccc6-429b-9031-5361d8a54248 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:56.218184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 73/140] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 14:59:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:56.223840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f21f20-8258-4b37-948a-ac46633d6543 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:56.328143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f21f20-8258-4b37-948a-ac46633d6543 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:56.345498 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2633622-4d6e-4fb6-b401-4e7d3c086b6d None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:56.346381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2633622-4d6e-4fb6-b401-4e7d3c086b6d None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:56.346868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2633622-4d6e-4fb6-b401-4e7d3c086b6d None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:56.352613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 74/141] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:56.361011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23ed1-fe8c-4561-81e4-00f16f29a4c4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:56.361607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23ed1-fe8c-4561-81e4-00f16f29a4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:56.362086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23ed1-fe8c-4561-81e4-00f16f29a4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:56.373061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bc76e-76aa-489e-b013-0231546de7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:56.373814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bc76e-76aa-489e-b013-0231546de7b6 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:56.374135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bc76e-76aa-489e-b013-0231546de7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:56.374397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bc76e-76aa-489e-b013-0231546de7b6 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:56.375854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bc76e-76aa-489e-b013-0231546de7b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:56.376702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bc76e-76aa-489e-b013-0231546de7b6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:56.396540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-dfe23ed1-fe8c-4561-81e4-00f16f29a4c4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:56.398116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-dfe23ed1-fe8c-4561-81e4-00f16f29a4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:56.415414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:56.418779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 68/142] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 14:59:56 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 14:59:56.500427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 75/143] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:56.543990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bc76e-76aa-489e-b013-0231546de7b6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:56.615893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:56.677519 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bc76e-76aa-489e-b013-0231546de7b6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:56.719132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79587f6-0a56-49e6-9ebd-dc6f2120a492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:56.719953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79587f6-0a56-49e6-9ebd-dc6f2120a492 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:56.720245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79587f6-0a56-49e6-9ebd-dc6f2120a492 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:56.720528 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79587f6-0a56-49e6-9ebd-dc6f2120a492 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:56.736030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79587f6-0a56-49e6-9ebd-dc6f2120a492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b79587f6-0a56-49e6-9ebd-dc6f2120a492 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:56.743720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:56.744599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:56.746183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 69/144] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:56.751544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe23ed1-fe8c-4561-81e4-00f16f29a4c4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:56.889977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe23ed1-fe8c-4561-81e4-00f16f29a4c4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:56.937259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a698c14-a5be-4c55-b62a-ed1ac596dfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:56.938321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a698c14-a5be-4c55-b62a-ed1ac596dfaf None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:56.938808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a698c14-a5be-4c55-b62a-ed1ac596dfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:56.939311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a698c14-a5be-4c55-b62a-ed1ac596dfaf None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:56.940367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a698c14-a5be-4c55-b62a-ed1ac596dfaf None admin] RBAC: Adding query filter params (name=member) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:56.940878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a698c14-a5be-4c55-b62a-ed1ac596dfaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:56.942463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a698c14-a5be-4c55-b62a-ed1ac596dfaf None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:56.954835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 76/145] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 14:59:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:56.960593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79587f6-0a56-49e6-9ebd-dc6f2120a492 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:57.089969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79587f6-0a56-49e6-9ebd-dc6f2120a492 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:57.111691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:57.125008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ed7d2-ba74-4ef9-a990-a965f705dfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:57.125909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ed7d2-ba74-4ef9-a990-a965f705dfd6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:57.126252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ed7d2-ba74-4ef9-a990-a965f705dfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:57.126545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ed7d2-ba74-4ef9-a990-a965f705dfd6 None admin] PATH_INFO: `/v3/users/49df8493343b42b0bb70939a8aa8aee1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:57.127797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ed7d2-ba74-4ef9-a990-a965f705dfd6 None admin] RBAC: Authorizing `identity:get_user(user_id=49df8493343b42b0bb70939a8aa8aee1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:57.129356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ed7d2-ba74-4ef9-a990-a965f705dfd6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:57.131205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 70/146] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:56 2022] GET /identity/v3/users/49df8493343b42b0bb70939a8aa8aee1 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:57.136597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a698c14-a5be-4c55-b62a-ed1ac596dfaf None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:57.152349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:57.255654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a698c14-a5be-4c55-b62a-ed1ac596dfaf None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:57.296729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dccb31-1e0e-47fe-8018-7549416c3ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:57.297392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dccb31-1e0e-47fe-8018-7549416c3ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:57.297682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dccb31-1e0e-47fe-8018-7549416c3ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:57.297964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dccb31-1e0e-47fe-8018-7549416c3ed3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:57.372480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dccb31-1e0e-47fe-8018-7549416c3ed3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:57.374022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dccb31-1e0e-47fe-8018-7549416c3ed3 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:57.382355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:57.481374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 77/147] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 14:59:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 14:59:57.486881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ed7d2-ba74-4ef9-a990-a965f705dfd6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:57.597572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ed7d2-ba74-4ef9-a990-a965f705dfd6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:57.634757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea51e0-b947-4475-83e0-732e04b98f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:57.635675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea51e0-b947-4475-83e0-732e04b98f8b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:57.635964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea51e0-b947-4475-83e0-732e04b98f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:57.636246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea51e0-b947-4475-83e0-732e04b98f8b None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:57.637451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea51e0-b947-4475-83e0-732e04b98f8b None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:57.638918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea51e0-b947-4475-83e0-732e04b98f8b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:57.640860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 71/148] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:57 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:57.646618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dccb31-1e0e-47fe-8018-7549416c3ed3 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:57.754369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dccb31-1e0e-47fe-8018-7549416c3ed3 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:57.761917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:57.794385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa989ec6-e743-437a-9acd-caa53b4f5c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:57.795068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa989ec6-e743-437a-9acd-caa53b4f5c2b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:57.795368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa989ec6-e743-437a-9acd-caa53b4f5c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:57.796184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa989ec6-e743-437a-9acd-caa53b4f5c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:57.797039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa989ec6-e743-437a-9acd-caa53b4f5c2b None admin] RBAC: Adding query filter params (role.id=6e32448ba01f41bcafb4cb0a839691fe, scope.project.id=a2c6e66812894558944eb8f6f156896d, user.id=49df8493343b42b0bb70939a8aa8aee1) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:57.797353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa989ec6-e743-437a-9acd-caa53b4f5c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:57.798825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa989ec6-e743-437a-9acd-caa53b4f5c2b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:57.807419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 78/149] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 14:59:57 2022] GET /identity/v3/role_assignments?user.id=49df8493343b42b0bb70939a8aa8aee1&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:58.024419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645b2f5-1975-4f48-81d0-de81c6725d16 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:58.024758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645b2f5-1975-4f48-81d0-de81c6725d16 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:58.024953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645b2f5-1975-4f48-81d0-de81c6725d16 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:58.026304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 72/150] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:58.040721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20923028-7710-47ae-a467-5f25392700e6 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:58.041020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20923028-7710-47ae-a467-5f25392700e6 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:58.041303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20923028-7710-47ae-a467-5f25392700e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:58.063327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-20923028-7710-47ae-a467-5f25392700e6 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:58.064542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-20923028-7710-47ae-a467-5f25392700e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:58.134537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 79/151] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:58.174363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645b2f5-1975-4f48-81d0-de81c6725d16 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:58.233774 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:58.302083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645b2f5-1975-4f48-81d0-de81c6725d16 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:58.344670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b093b71-9111-4a90-bb4f-0ce01d4a67ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:58.345503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b093b71-9111-4a90-bb4f-0ce01d4a67ef None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:58.345791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b093b71-9111-4a90-bb4f-0ce01d4a67ef None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:58.346074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b093b71-9111-4a90-bb4f-0ce01d4a67ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:58.357029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b093b71-9111-4a90-bb4f-0ce01d4a67ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-9b093b71-9111-4a90-bb4f-0ce01d4a67ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:58.365348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:58.366226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:58.367009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 73/152] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 14:59:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 14:59:58.373510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20923028-7710-47ae-a467-5f25392700e6 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:58.484470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:58.487324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20923028-7710-47ae-a467-5f25392700e6 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:58.531120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08da60-afca-42ef-8f1f-393fb12f23ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:58.531806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08da60-afca-42ef-8f1f-393fb12f23ed None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:58.532092 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08da60-afca-42ef-8f1f-393fb12f23ed None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:58.532373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d08da60-afca-42ef-8f1f-393fb12f23ed None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:58.533138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08da60-afca-42ef-8f1f-393fb12f23ed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:58.533449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08da60-afca-42ef-8f1f-393fb12f23ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:58.534837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d08da60-afca-42ef-8f1f-393fb12f23ed None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:58.546294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 80/153] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 14:59:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:58.551760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b093b71-9111-4a90-bb4f-0ce01d4a67ef None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:58.674569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b093b71-9111-4a90-bb4f-0ce01d4a67ef None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:58.685772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:58.720294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8bf19-560e-4a5a-9f56-4a9f2725945c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:58.721276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8bf19-560e-4a5a-9f56-4a9f2725945c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:58.721563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8bf19-560e-4a5a-9f56-4a9f2725945c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:58.721889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8bf19-560e-4a5a-9f56-4a9f2725945c None admin] PATH_INFO: `/v3/users/49df8493343b42b0bb70939a8aa8aee1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:58.723133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8bf19-560e-4a5a-9f56-4a9f2725945c None admin] RBAC: Authorizing `identity:get_user(user_id=49df8493343b42b0bb70939a8aa8aee1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:58.724615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8bf19-560e-4a5a-9f56-4a9f2725945c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:58.726698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 74/154] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:58 2022] GET /identity/v3/users/49df8493343b42b0bb70939a8aa8aee1 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:58.731803 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08da60-afca-42ef-8f1f-393fb12f23ed None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:58.854709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d08da60-afca-42ef-8f1f-393fb12f23ed None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:58.900931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ef681-64d6-4198-87e0-d65462e65872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:58.901589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ef681-64d6-4198-87e0-d65462e65872 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:58.901870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ef681-64d6-4198-87e0-d65462e65872 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:58.902147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0ef681-64d6-4198-87e0-d65462e65872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:58.931793 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b604627-0482-4b81-a457-50b51af5a40e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:58.932411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b604627-0482-4b81-a457-50b51af5a40e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:58.932889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b604627-0482-4b81-a457-50b51af5a40e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:58.934715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 75/155] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:58.941793 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46fbac-6f86-4bd9-bf9e-bdd2d7bc49d6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:58.942331 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46fbac-6f86-4bd9-bf9e-bdd2d7bc49d6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:58.942820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb46fbac-6f86-4bd9-bf9e-bdd2d7bc49d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:58.949856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:58.966109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-eb46fbac-6f86-4bd9-bf9e-bdd2d7bc49d6 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:58.971850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eb46fbac-6f86-4bd9-bf9e-bdd2d7bc49d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:59.023338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ef681-64d6-4198-87e0-d65462e65872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:59.024827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0ef681-64d6-4198-87e0-d65462e65872 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:59.114052 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 76/156] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:59.153464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46fbac-6f86-4bd9-bf9e-bdd2d7bc49d6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:59.255855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 81/157] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 14:59:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 14:59:59.266732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ef681-64d6-4198-87e0-d65462e65872 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:59.344292 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb46fbac-6f86-4bd9-bf9e-bdd2d7bc49d6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:59.353832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:59.392883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c55763-13a6-47b8-84ff-2a358b33e585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:59.393694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c55763-13a6-47b8-84ff-2a358b33e585 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:59.393982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c55763-13a6-47b8-84ff-2a358b33e585 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:59.394263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c55763-13a6-47b8-84ff-2a358b33e585 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:59.396267 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:59.409156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c55763-13a6-47b8-84ff-2a358b33e585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-a4c55763-13a6-47b8-84ff-2a358b33e585 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 14:59:59.417025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 14:59:59.417986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 14:59:59.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 14:59:59.424080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 77/158] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 14:59:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 14:59:59.429570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c55763-13a6-47b8-84ff-2a358b33e585 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:59.464103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0ef681-64d6-4198-87e0-d65462e65872 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:59.507691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b5ea6-1eb5-41a4-bc41-df3833619835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:59.508556 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5ea6-1eb5-41a4-bc41-df3833619835 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:59.509011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5ea6-1eb5-41a4-bc41-df3833619835 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:59.509425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b5ea6-1eb5-41a4-bc41-df3833619835 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:59.510764 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b5ea6-1eb5-41a4-bc41-df3833619835 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:59.512781 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b5ea6-1eb5-41a4-bc41-df3833619835 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:59.515902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 82/159] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 14:59:59 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:59.524277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b5ea6-1eb5-41a4-bc41-df3833619835 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:59.621520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c55763-13a6-47b8-84ff-2a358b33e585 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:59.637119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b5ea6-1eb5-41a4-bc41-df3833619835 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:59.674482 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a18a3-00ee-4df2-bd74-0051a2b65fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:59.675681 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a18a3-00ee-4df2-bd74-0051a2b65fda None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:59.676174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a18a3-00ee-4df2-bd74-0051a2b65fda None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:59.676644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a18a3-00ee-4df2-bd74-0051a2b65fda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:59.677698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a18a3-00ee-4df2-bd74-0051a2b65fda None admin] RBAC: Adding query filter params (role.id=4301bced0259446cafb79118a53a015e, scope.project.id=a2c6e66812894558944eb8f6f156896d, user.id=49df8493343b42b0bb70939a8aa8aee1) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:59.678219 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a18a3-00ee-4df2-bd74-0051a2b65fda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:59.679901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:59.680580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a18a3-00ee-4df2-bd74-0051a2b65fda None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:59.687493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 83/160] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 14:59:59 2022] GET /identity/v3/role_assignments?user.id=49df8493343b42b0bb70939a8aa8aee1&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=4301bced0259446cafb79118a53a015e => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 14:59:59.689229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced28762-6c8e-4d54-9287-dcefff1e074d None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:59.689888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced28762-6c8e-4d54-9287-dcefff1e074d None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:59.690336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced28762-6c8e-4d54-9287-dcefff1e074d None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:59.692095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 84/161] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:59.697868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523d0b3-42a5-4b4c-8d73-bf3753549b54 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:59.698378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523d0b3-42a5-4b4c-8d73-bf3753549b54 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:59.698801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523d0b3-42a5-4b4c-8d73-bf3753549b54 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:59.701953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 85/162] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 14:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 14:59:59.703833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9241b-c1a8-4619-81ab-6b0639d32738 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:59.704482 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9241b-c1a8-4619-81ab-6b0639d32738 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:59.704982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9241b-c1a8-4619-81ab-6b0639d32738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:59.725799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-96b9241b-c1a8-4619-81ab-6b0639d32738 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:59.727462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-96b9241b-c1a8-4619-81ab-6b0639d32738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:59.755878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e6bef-03e7-4faa-84a1-3776b556f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:59.756581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e6bef-03e7-4faa-84a1-3776b556f2fa None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:59.756890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e6bef-03e7-4faa-84a1-3776b556f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:59.757156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e6bef-03e7-4faa-84a1-3776b556f2fa None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:59.758114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e6bef-03e7-4faa-84a1-3776b556f2fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 14:59:59.758581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e6bef-03e7-4faa-84a1-3776b556f2fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:59.760174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e6bef-03e7-4faa-84a1-3776b556f2fa None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 14:59:59.785096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 78/163] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 14:59:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 14:59:59.786656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e5ed1-9e28-45bb-bce6-6709f3d76f42 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:59.787128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e5ed1-9e28-45bb-bce6-6709f3d76f42 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:59.787567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7e5ed1-9e28-45bb-bce6-6709f3d76f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:59.811656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 86/164] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:59.813611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9241b-c1a8-4619-81ab-6b0639d32738 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:59.836176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8c7e5ed1-9e28-45bb-bce6-6709f3d76f42 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 14:59:59.837406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8c7e5ed1-9e28-45bb-bce6-6709f3d76f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 14:59:59.890586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:59.951347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9241b-c1a8-4619-81ab-6b0639d32738 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 14:59:59.978592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 79/165] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 14:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 14:59:59.979765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e5ed1-9e28-45bb-bce6-6709f3d76f42 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 14:59:59.981593 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 14:59:59.993893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a27bec-6fdb-4a78-9b90-63856805adc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 14:59:59.995184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a27bec-6fdb-4a78-9b90-63856805adc2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 14:59:59.995586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a27bec-6fdb-4a78-9b90-63856805adc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 14:59:59.995958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a27bec-6fdb-4a78-9b90-63856805adc2 None admin] PATH_INFO: `/v3/users/013c7ef73b0e4bc0b335f79c4f2a2ffa` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 14:59:59.997460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a27bec-6fdb-4a78-9b90-63856805adc2 None admin] RBAC: Authorizing `identity:get_user(user_id=013c7ef73b0e4bc0b335f79c4f2a2ffa)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 14:59:59.999239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a27bec-6fdb-4a78-9b90-63856805adc2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:00.001756 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 87/166] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 14:59:59 2022] GET /identity/v3/users/013c7ef73b0e4bc0b335f79c4f2a2ffa => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:00.009881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a27bec-6fdb-4a78-9b90-63856805adc2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:00.098270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7e5ed1-9e28-45bb-bce6-6709f3d76f42 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:00.134887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b0cf1-2f66-4000-b8b8-d263f0cf25b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:00.136204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b0cf1-2f66-4000-b8b8-d263f0cf25b1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.136692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b0cf1-2f66-4000-b8b8-d263f0cf25b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.137169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6b0cf1-2f66-4000-b8b8-d263f0cf25b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:00.139697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a27bec-6fdb-4a78-9b90-63856805adc2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:00.148824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6b0cf1-2f66-4000-b8b8-d263f0cf25b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-5a6b0cf1-2f66-4000-b8b8-d263f0cf25b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:00.157094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:00.157944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:00.158980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:00.160912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 80/167] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 14:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:00.161845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b0cf1-2f66-4000-b8b8-d263f0cf25b1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:00.184623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:00.209820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411741e-b61b-4307-8f93-106a365a3858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:00.210778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411741e-b61b-4307-8f93-106a365a3858 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.215073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411741e-b61b-4307-8f93-106a365a3858 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.215073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411741e-b61b-4307-8f93-106a365a3858 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:00.224787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411741e-b61b-4307-8f93-106a365a3858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-d411741e-b61b-4307-8f93-106a365a3858 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:00.236287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:00.237336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:00.238183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 88/168] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:00.238910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411741e-b61b-4307-8f93-106a365a3858 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:00.302539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:00.357559 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411741e-b61b-4307-8f93-106a365a3858 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:00.377768 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6b0cf1-2f66-4000-b8b8-d263f0cf25b1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:00.406798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758e3a3-fe0d-4d1f-9b31-dddaefd5462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:00.407508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758e3a3-fe0d-4d1f-9b31-dddaefd5462f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.407794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758e3a3-fe0d-4d1f-9b31-dddaefd5462f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.408074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758e3a3-fe0d-4d1f-9b31-dddaefd5462f None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:00.409648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758e3a3-fe0d-4d1f-9b31-dddaefd5462f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:00.409961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758e3a3-fe0d-4d1f-9b31-dddaefd5462f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:00.411379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758e3a3-fe0d-4d1f-9b31-dddaefd5462f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:00.420092 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 89/169] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:00.420982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758e3a3-fe0d-4d1f-9b31-dddaefd5462f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:00.476685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1861464-7db4-4b20-823c-e8b28d19993a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:00.477569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1861464-7db4-4b20-823c-e8b28d19993a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.477858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1861464-7db4-4b20-823c-e8b28d19993a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.478136 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1861464-7db4-4b20-823c-e8b28d19993a None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:00.483780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1861464-7db4-4b20-823c-e8b28d19993a None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:00.485276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1861464-7db4-4b20-823c-e8b28d19993a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:00.487077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 81/170] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:00 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:00.487797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1861464-7db4-4b20-823c-e8b28d19993a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:00.550848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758e3a3-fe0d-4d1f-9b31-dddaefd5462f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:00.576724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:00.615327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecf3cd-e2f1-45ab-9c3b-f7ada68b2c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:00.616939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecf3cd-e2f1-45ab-9c3b-f7ada68b2c9a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.618174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecf3cd-e2f1-45ab-9c3b-f7ada68b2c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.618454 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cecf3cd-e2f1-45ab-9c3b-f7ada68b2c9a None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:00.619251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecf3cd-e2f1-45ab-9c3b-f7ada68b2c9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:00.619561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecf3cd-e2f1-45ab-9c3b-f7ada68b2c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:00.620968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cecf3cd-e2f1-45ab-9c3b-f7ada68b2c9a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:00.634691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 90/171] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:00.634691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecf3cd-e2f1-45ab-9c3b-f7ada68b2c9a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:00.643497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:00.657448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1861464-7db4-4b20-823c-e8b28d19993a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:00.695868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44feb0-2e85-41fb-89e8-3af49e815be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:00.697108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44feb0-2e85-41fb-89e8-3af49e815be2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.697578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44feb0-2e85-41fb-89e8-3af49e815be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.698000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44feb0-2e85-41fb-89e8-3af49e815be2 None admin] PATH_INFO: `/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:00.699376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44feb0-2e85-41fb-89e8-3af49e815be2 None admin] RBAC: Authorizing `identity:get_user(user_id=7b7d0fd6414b417ab8bf3499e0f4961b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:00.701183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44feb0-2e85-41fb-89e8-3af49e815be2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:00.703610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 82/172] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:00 2022] GET /identity/v3/users/7b7d0fd6414b417ab8bf3499e0f4961b => generated 247 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:00.705030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35d30-91e3-4d04-8bc4-ea2bde2e6ade None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.705461 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35d30-91e3-4d04-8bc4-ea2bde2e6ade None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.705878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e35d30-91e3-4d04-8bc4-ea2bde2e6ade None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:00.707346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 83/173] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:00.711953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e35d30-91e3-4d04-8bc4-ea2bde2e6ade None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:00.747870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cecf3cd-e2f1-45ab-9c3b-f7ada68b2c9a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:00.782501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b95b9-22f5-4e53-a416-8790c8d035ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:00.783437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b95b9-22f5-4e53-a416-8790c8d035ad None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.783719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b95b9-22f5-4e53-a416-8790c8d035ad None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.784078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b95b9-22f5-4e53-a416-8790c8d035ad None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d/users/013c7ef73b0e4bc0b335f79c4f2a2ffa/roles/dd4438dfc8ea468db528cc4cda58f3b8` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:00.785985 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b95b9-22f5-4e53-a416-8790c8d035ad None admin] RBAC: Authorizing `identity:create_grant(project_id=a2c6e66812894558944eb8f6f156896d, user_id=013c7ef73b0e4bc0b335f79c4f2a2ffa, role_id=dd4438dfc8ea468db528cc4cda58f3b8)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:00.787495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b95b9-22f5-4e53-a416-8790c8d035ad None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:00.804804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 91/174] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:00:00 2022] PUT /identity/v3/projects/a2c6e66812894558944eb8f6f156896d/users/013c7ef73b0e4bc0b335f79c4f2a2ffa/roles/dd4438dfc8ea468db528cc4cda58f3b8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:00.805808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b95b9-22f5-4e53-a416-8790c8d035ad None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:00.844024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e35d30-91e3-4d04-8bc4-ea2bde2e6ade None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:00.851980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:00.935421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b95b9-22f5-4e53-a416-8790c8d035ad None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:00.968497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad4d0a-0f2c-4776-9a4e-19b01438872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:00.969326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad4d0a-0f2c-4776-9a4e-19b01438872c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.969687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad4d0a-0f2c-4776-9a4e-19b01438872c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.969884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad4d0a-0f2c-4776-9a4e-19b01438872c None admin] PATH_INFO: `/v3/users/49df8493343b42b0bb70939a8aa8aee1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:00.976023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad4d0a-0f2c-4776-9a4e-19b01438872c None admin] RBAC: Authorizing `identity:get_user(user_id=49df8493343b42b0bb70939a8aa8aee1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:00.977581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad4d0a-0f2c-4776-9a4e-19b01438872c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:00.977950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efe382-7ccd-4130-b984-ade45dfba2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:00.978796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efe382-7ccd-4130-b984-ade45dfba2da None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.979252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efe382-7ccd-4130-b984-ade45dfba2da None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.979377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 84/175] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:00 2022] GET /identity/v3/users/49df8493343b42b0bb70939a8aa8aee1 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:00.979913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efe382-7ccd-4130-b984-ade45dfba2da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:00.980581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d499f8-d7b1-4957-96a5-070629f12f13 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:00.980872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d499f8-d7b1-4957-96a5-070629f12f13 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:00.981153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d499f8-d7b1-4957-96a5-070629f12f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.011083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c0d499f8-d7b1-4957-96a5-070629f12f13 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:01.013642 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c0d499f8-d7b1-4957-96a5-070629f12f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:01.034223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:01.070536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efe382-7ccd-4130-b984-ade45dfba2da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:01.072912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efe382-7ccd-4130-b984-ade45dfba2da None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:01.098141 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 85/176] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:01.098802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d499f8-d7b1-4957-96a5-070629f12f13 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:01.111203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:01.177498 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 92/177] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:01.178825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13c948-89a1-4cce-9539-a4e61907d9e7 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.179144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13c948-89a1-4cce-9539-a4e61907d9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.179424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d13c948-89a1-4cce-9539-a4e61907d9e7 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.183091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 93/178] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:01.183091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1575d-0ed4-41c7-8b2b-1388242ba470 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.183091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1575d-0ed4-41c7-8b2b-1388242ba470 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.183091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1575d-0ed4-41c7-8b2b-1388242ba470 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.183800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 94/179] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:01.188025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1575d-0ed4-41c7-8b2b-1388242ba470 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:01.221637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d499f8-d7b1-4957-96a5-070629f12f13 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:01.258006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061967e4-eb5a-442c-86af-9a65dd01ac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:01.258657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061967e4-eb5a-442c-86af-9a65dd01ac64 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.258945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061967e4-eb5a-442c-86af-9a65dd01ac64 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.259253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061967e4-eb5a-442c-86af-9a65dd01ac64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.284669 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:01.304819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1575d-0ed4-41c7-8b2b-1388242ba470 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:01.330062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061967e4-eb5a-442c-86af-9a65dd01ac64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:01.331565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061967e4-eb5a-442c-86af-9a65dd01ac64 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:01.341732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753c64-aef4-4e4c-965c-012b4c403e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:01.343089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753c64-aef4-4e4c-965c-012b4c403e49 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.343362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753c64-aef4-4e4c-965c-012b4c403e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.343639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b753c64-aef4-4e4c-965c-012b4c403e49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.346624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:01.355532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b753c64-aef4-4e4c-965c-012b4c403e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-9b753c64-aef4-4e4c-965c-012b4c403e49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:01.363826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:01.364733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:01.365539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 95/180] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:00:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:01.366200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753c64-aef4-4e4c-965c-012b4c403e49 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:01.440092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 86/181] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:01.440092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcec91f-3f93-4535-973f-8575e9e2a082 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.440092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcec91f-3f93-4535-973f-8575e9e2a082 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.440092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcec91f-3f93-4535-973f-8575e9e2a082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.459156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2fcec91f-3f93-4535-973f-8575e9e2a082 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:01.460616 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2fcec91f-3f93-4535-973f-8575e9e2a082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:01.486765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b753c64-aef4-4e4c-965c-012b4c403e49 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:01.510572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:01.521695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bbf255-34db-4eb0-8664-fbc9e55e6ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:01.522559 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbf255-34db-4eb0-8664-fbc9e55e6ccd None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.522843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbf255-34db-4eb0-8664-fbc9e55e6ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.523166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bbf255-34db-4eb0-8664-fbc9e55e6ccd None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.524360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbf255-34db-4eb0-8664-fbc9e55e6ccd None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:01.525810 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bbf255-34db-4eb0-8664-fbc9e55e6ccd None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:01.527795 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 96/182] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:01 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:01.529002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b298c9-f948-431e-a591-fe429bb834f1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.529290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b298c9-f948-431e-a591-fe429bb834f1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.529568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b298c9-f948-431e-a591-fe429bb834f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.537090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 87/183] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:01.538121 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcec91f-3f93-4535-973f-8575e9e2a082 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:01.547649 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-93b298c9-f948-431e-a591-fe429bb834f1 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:01.548852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-93b298c9-f948-431e-a591-fe429bb834f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:01.609765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:01.627068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 97/184] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:01.627068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b298c9-f948-431e-a591-fe429bb834f1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:01.660129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcec91f-3f93-4535-973f-8575e9e2a082 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:01.696308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918cef-f743-4a8a-9514-bbbceade5c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:01.696978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918cef-f743-4a8a-9514-bbbceade5c6b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.697263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918cef-f743-4a8a-9514-bbbceade5c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.697537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918cef-f743-4a8a-9514-bbbceade5c6b None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.698284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5918cef-f743-4a8a-9514-bbbceade5c6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:01.698596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5918cef-f743-4a8a-9514-bbbceade5c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:01.700018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5918cef-f743-4a8a-9514-bbbceade5c6b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:01.709503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 88/185] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:01.710352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918cef-f743-4a8a-9514-bbbceade5c6b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:01.733466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b298c9-f948-431e-a591-fe429bb834f1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:01.758433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:01.771418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c3c38-0c2f-451f-b4ea-acc29822a87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:01.772388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c3c38-0c2f-451f-b4ea-acc29822a87d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.772673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c3c38-0c2f-451f-b4ea-acc29822a87d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.772951 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c3c38-0c2f-451f-b4ea-acc29822a87d None admin] PATH_INFO: `/v3/projects/e59fd922239b4521b0c65581d18ac618` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.774235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c3c38-0c2f-451f-b4ea-acc29822a87d None admin] RBAC: Authorizing `identity:get_project(project_id=e59fd922239b4521b0c65581d18ac618)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:01.775848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c3c38-0c2f-451f-b4ea-acc29822a87d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:01.777938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 98/186] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:01 2022] GET /identity/v3/projects/e59fd922239b4521b0c65581d18ac618 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:01.778816 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c3c38-0c2f-451f-b4ea-acc29822a87d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:01.818272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918cef-f743-4a8a-9514-bbbceade5c6b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:01.844123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:01.856765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d07c-65aa-41f3-a4a9-085eaee6e6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:01.857423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d07c-65aa-41f3-a4a9-085eaee6e6f9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.857708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d07c-65aa-41f3-a4a9-085eaee6e6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.858499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572d07c-65aa-41f3-a4a9-085eaee6e6f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.859373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572d07c-65aa-41f3-a4a9-085eaee6e6f9 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=a2c6e66812894558944eb8f6f156896d, user.id=7b7d0fd6414b417ab8bf3499e0f4961b) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:01.859686 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572d07c-65aa-41f3-a4a9-085eaee6e6f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:01.861118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572d07c-65aa-41f3-a4a9-085eaee6e6f9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:01.879084 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 89/187] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:01 2022] GET /identity/v3/role_assignments?user.id=7b7d0fd6414b417ab8bf3499e0f4961b&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:01.879084 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d07c-65aa-41f3-a4a9-085eaee6e6f9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:01.900983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c3c38-0c2f-451f-b4ea-acc29822a87d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:01.952848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b22238-3df6-428b-ba0e-a40ee6d58c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:01.954239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b22238-3df6-428b-ba0e-a40ee6d58c94 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:01.955115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b22238-3df6-428b-ba0e-a40ee6d58c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:01.955884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b22238-3df6-428b-ba0e-a40ee6d58c94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:01.957437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b22238-3df6-428b-ba0e-a40ee6d58c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:01.959428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b22238-3df6-428b-ba0e-a40ee6d58c94 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:01.962551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 99/188] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:01.964680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b22238-3df6-428b-ba0e-a40ee6d58c94 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:01.997919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572d07c-65aa-41f3-a4a9-085eaee6e6f9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:02.010667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:02.043428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356782b1-401f-405c-b163-cfa2fc1b27cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:02.044254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356782b1-401f-405c-b163-cfa2fc1b27cb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:02.044538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356782b1-401f-405c-b163-cfa2fc1b27cb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:02.044813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356782b1-401f-405c-b163-cfa2fc1b27cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:02.060514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356782b1-401f-405c-b163-cfa2fc1b27cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-356782b1-401f-405c-b163-cfa2fc1b27cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:02.070225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:02.071145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:02.071936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:02.077981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 90/189] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:00:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:02.077981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356782b1-401f-405c-b163-cfa2fc1b27cb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:02.103894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b22238-3df6-428b-ba0e-a40ee6d58c94 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:02.146958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:02.153605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cfc6a-e24e-4896-b185-7aac62aeb716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:02.155429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cfc6a-e24e-4896-b185-7aac62aeb716 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:02.155934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cfc6a-e24e-4896-b185-7aac62aeb716 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:02.156464 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5cfc6a-e24e-4896-b185-7aac62aeb716 None admin] PATH_INFO: `/v3/users/197283397a3e4623bbeab4274197d919` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:02.170770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cfc6a-e24e-4896-b185-7aac62aeb716 None admin] RBAC: Authorizing `identity:get_user(user_id=197283397a3e4623bbeab4274197d919)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:02.172309 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5cfc6a-e24e-4896-b185-7aac62aeb716 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:02.174053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 100/190] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:01 2022] GET /identity/v3/users/197283397a3e4623bbeab4274197d919 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:02.174739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cfc6a-e24e-4896-b185-7aac62aeb716 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:02.189946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356782b1-401f-405c-b163-cfa2fc1b27cb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:02.231305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c88707-883d-493e-8fb7-8bcab2da63f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:02.232127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c88707-883d-493e-8fb7-8bcab2da63f1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:02.232245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c88707-883d-493e-8fb7-8bcab2da63f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:02.232515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c88707-883d-493e-8fb7-8bcab2da63f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:02.233478 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c88707-883d-493e-8fb7-8bcab2da63f1 None admin] RBAC: Adding query filter params (role.id=6e32448ba01f41bcafb4cb0a839691fe, scope.project.id=e59fd922239b4521b0c65581d18ac618, user.id=49df8493343b42b0bb70939a8aa8aee1) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:02.233761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c88707-883d-493e-8fb7-8bcab2da63f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:02.235231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c88707-883d-493e-8fb7-8bcab2da63f1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:02.240953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 91/191] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:02 2022] GET /identity/v3/role_assignments?user.id=49df8493343b42b0bb70939a8aa8aee1&scope.project.id=e59fd922239b4521b0c65581d18ac618&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:02.241729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c88707-883d-493e-8fb7-8bcab2da63f1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:02.289225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5cfc6a-e24e-4896-b185-7aac62aeb716 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:02.291395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:02.330790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4087d4-4db8-4d9b-9397-05dfe2856262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:02.331424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4087d4-4db8-4d9b-9397-05dfe2856262 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:02.331714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4087d4-4db8-4d9b-9397-05dfe2856262 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:02.331996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4087d4-4db8-4d9b-9397-05dfe2856262 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:02.332742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4087d4-4db8-4d9b-9397-05dfe2856262 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:02.334243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4087d4-4db8-4d9b-9397-05dfe2856262 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:02.362155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 101/192] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:00:02 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:00:02.364421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4087d4-4db8-4d9b-9397-05dfe2856262 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:02.375242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c88707-883d-493e-8fb7-8bcab2da63f1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:02.416405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a74f7-039d-4e7f-9475-cb3bd9818b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:02.417323 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a74f7-039d-4e7f-9475-cb3bd9818b13 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:02.417514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a74f7-039d-4e7f-9475-cb3bd9818b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:02.417770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a74f7-039d-4e7f-9475-cb3bd9818b13 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:02.419185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a74f7-039d-4e7f-9475-cb3bd9818b13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:02.419185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a74f7-039d-4e7f-9475-cb3bd9818b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:02.421888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a74f7-039d-4e7f-9475-cb3bd9818b13 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:02.431506 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 92/193] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:02.433079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:02.440631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a74f7-039d-4e7f-9475-cb3bd9818b13 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:02.507453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4087d4-4db8-4d9b-9397-05dfe2856262 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:02.552342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af93e5-4112-437e-97d5-d075d91e1803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:02.552948 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af93e5-4112-437e-97d5-d075d91e1803 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:02.553232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af93e5-4112-437e-97d5-d075d91e1803 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:02.553516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af93e5-4112-437e-97d5-d075d91e1803 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:02.565843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:02.577305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a74f7-039d-4e7f-9475-cb3bd9818b13 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:02.627534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee955383-c21f-4fbe-a59c-4dcebc913c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:02.628420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee955383-c21f-4fbe-a59c-4dcebc913c1a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:02.628701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee955383-c21f-4fbe-a59c-4dcebc913c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:02.629208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee955383-c21f-4fbe-a59c-4dcebc913c1a None admin] PATH_INFO: `/v3/users/197283397a3e4623bbeab4274197d919` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:02.630409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee955383-c21f-4fbe-a59c-4dcebc913c1a None admin] RBAC: Authorizing `identity:get_user(user_id=197283397a3e4623bbeab4274197d919)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:02.632045 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee955383-c21f-4fbe-a59c-4dcebc913c1a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:02.633915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 93/194] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:02 2022] GET /identity/v3/users/197283397a3e4623bbeab4274197d919 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:02.639611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee955383-c21f-4fbe-a59c-4dcebc913c1a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:02.654161 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af93e5-4112-437e-97d5-d075d91e1803 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:02.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af93e5-4112-437e-97d5-d075d91e1803 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:02.722853 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:02.760443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee955383-c21f-4fbe-a59c-4dcebc913c1a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:02.804464 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 102/195] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:02.805344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5cb27-1706-490d-ae6d-b0dfa58d88df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:02.806011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5cb27-1706-490d-ae6d-b0dfa58d88df None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:02.806302 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5cb27-1706-490d-ae6d-b0dfa58d88df None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:02.806592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5cb27-1706-490d-ae6d-b0dfa58d88df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:02.810411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af93e5-4112-437e-97d5-d075d91e1803 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:02.852306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:02.904191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5cb27-1706-490d-ae6d-b0dfa58d88df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:02.906505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5cb27-1706-490d-ae6d-b0dfa58d88df None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:02.930638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af93e5-4112-437e-97d5-d075d91e1803 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:02.980318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba8588-4137-4637-8794-eab3372a5058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:02.981538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba8588-4137-4637-8794-eab3372a5058 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:02.982000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba8588-4137-4637-8794-eab3372a5058 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:02.982500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba8588-4137-4637-8794-eab3372a5058 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:02.984972 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba8588-4137-4637-8794-eab3372a5058 None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:02.986671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba8588-4137-4637-8794-eab3372a5058 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:02.988930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 103/196] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:02 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:02.995538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba8588-4137-4637-8794-eab3372a5058 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:03.012538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:03.106958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 94/197] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:03.109113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4d52c-783a-4346-aa42-0f493a531e90 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:03.109397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4d52c-783a-4346-aa42-0f493a531e90 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:03.109681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba4d52c-783a-4346-aa42-0f493a531e90 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:03.115190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 95/198] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:03.116012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4d52c-783a-4346-aa42-0f493a531e90 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:03.165179 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba8588-4137-4637-8794-eab3372a5058 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:03.198401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:03.224877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43429c1a-3ba9-472d-930d-7f75f6cee177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:03.225545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43429c1a-3ba9-472d-930d-7f75f6cee177 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:03.225833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43429c1a-3ba9-472d-930d-7f75f6cee177 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:03.226112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43429c1a-3ba9-472d-930d-7f75f6cee177 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:03.227436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43429c1a-3ba9-472d-930d-7f75f6cee177 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=96f8d6ac74154c299166a6e2bf295a72, user.id=197283397a3e4623bbeab4274197d919) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:03.227763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43429c1a-3ba9-472d-930d-7f75f6cee177 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:03.229527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43429c1a-3ba9-472d-930d-7f75f6cee177 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:03.239153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 104/199] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:02 2022] GET /identity/v3/role_assignments?user.id=197283397a3e4623bbeab4274197d919&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:03.240706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ab34f-b6fe-4dce-803b-447b993c9901 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:03.241212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ab34f-b6fe-4dce-803b-447b993c9901 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:03.241700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0ab34f-b6fe-4dce-803b-447b993c9901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:03.248123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba4d52c-783a-4346-aa42-0f493a531e90 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:03.275217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-eb0ab34f-b6fe-4dce-803b-447b993c9901 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:03.278496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-eb0ab34f-b6fe-4dce-803b-447b993c9901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:03.309974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5be5e-f353-43b6-9f3f-88741996d6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:03.310934 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5be5e-f353-43b6-9f3f-88741996d6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:03.311359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5be5e-f353-43b6-9f3f-88741996d6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:03.311638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5be5e-f353-43b6-9f3f-88741996d6c5 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:03.312920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5be5e-f353-43b6-9f3f-88741996d6c5 None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:03.319094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5be5e-f353-43b6-9f3f-88741996d6c5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:03.319094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 96/200] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:03 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:03.323543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5be5e-f353-43b6-9f3f-88741996d6c5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:03.448450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5be5e-f353-43b6-9f3f-88741996d6c5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:03.456823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 105/201] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:03.480220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ab34f-b6fe-4dce-803b-447b993c9901 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:03.483818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:03.489536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f3f41-4f83-4b20-b88e-ea9f0e7f8172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:03.490799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f3f41-4f83-4b20-b88e-ea9f0e7f8172 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:03.491133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f3f41-4f83-4b20-b88e-ea9f0e7f8172 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:03.491428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f3f41-4f83-4b20-b88e-ea9f0e7f8172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:03.492639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f3f41-4f83-4b20-b88e-ea9f0e7f8172 None admin] RBAC: Adding query filter params (role.id=4301bced0259446cafb79118a53a015e, scope.project.id=96f8d6ac74154c299166a6e2bf295a72, user.id=197283397a3e4623bbeab4274197d919) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:03.492980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f3f41-4f83-4b20-b88e-ea9f0e7f8172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:03.494266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:03.494919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f3f41-4f83-4b20-b88e-ea9f0e7f8172 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:03.502564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 97/202] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:03 2022] GET /identity/v3/role_assignments?user.id=197283397a3e4623bbeab4274197d919&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=4301bced0259446cafb79118a53a015e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:03.596408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0ab34f-b6fe-4dce-803b-447b993c9901 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:03.634248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477005-db27-4054-8b14-6a6aeec85641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:03.635095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477005-db27-4054-8b14-6a6aeec85641 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:03.635385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477005-db27-4054-8b14-6a6aeec85641 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:03.635666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477005-db27-4054-8b14-6a6aeec85641 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:03.648124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477005-db27-4054-8b14-6a6aeec85641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-b5477005-db27-4054-8b14-6a6aeec85641 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:03.656906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:03.658338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:03.659171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:03.663098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 106/203] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:03.664513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f3f41-4f83-4b20-b88e-ea9f0e7f8172 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:03.793794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f3f41-4f83-4b20-b88e-ea9f0e7f8172 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:03.837668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797e2a2-5ae6-49bc-b65b-05ff64850677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:03.838346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797e2a2-5ae6-49bc-b65b-05ff64850677 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:03.838637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797e2a2-5ae6-49bc-b65b-05ff64850677 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:03.838918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797e2a2-5ae6-49bc-b65b-05ff64850677 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:03.839710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797e2a2-5ae6-49bc-b65b-05ff64850677 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:03.840028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797e2a2-5ae6-49bc-b65b-05ff64850677 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:03.841428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797e2a2-5ae6-49bc-b65b-05ff64850677 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:03.850699 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 98/204] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:03.856108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477005-db27-4054-8b14-6a6aeec85641 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:03.956436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:03.991912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477005-db27-4054-8b14-6a6aeec85641 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:04.026557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c94522-8d29-4bc6-9066-e94c66a9885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:04.027467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c94522-8d29-4bc6-9066-e94c66a9885a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:04.027751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c94522-8d29-4bc6-9066-e94c66a9885a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:04.028026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c94522-8d29-4bc6-9066-e94c66a9885a None admin] PATH_INFO: `/v3/users/013c7ef73b0e4bc0b335f79c4f2a2ffa` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:04.029211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c94522-8d29-4bc6-9066-e94c66a9885a None admin] RBAC: Authorizing `identity:get_user(user_id=013c7ef73b0e4bc0b335f79c4f2a2ffa)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:04.030786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c94522-8d29-4bc6-9066-e94c66a9885a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:04.036899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 107/205] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:03 2022] GET /identity/v3/users/013c7ef73b0e4bc0b335f79c4f2a2ffa => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:04.038251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797e2a2-5ae6-49bc-b65b-05ff64850677 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:04.108775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:04.169186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797e2a2-5ae6-49bc-b65b-05ff64850677 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:04.223097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b28b2-fb99-47a6-9157-bc1ff8222351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:04.223769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b28b2-fb99-47a6-9157-bc1ff8222351 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:04.224053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b28b2-fb99-47a6-9157-bc1ff8222351 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:04.224333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b28b2-fb99-47a6-9157-bc1ff8222351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:04.291580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b28b2-fb99-47a6-9157-bc1ff8222351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:04.293064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b28b2-fb99-47a6-9157-bc1ff8222351 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:04.365409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:04.422587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 99/206] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:04.428007 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c94522-8d29-4bc6-9066-e94c66a9885a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:04.537814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c94522-8d29-4bc6-9066-e94c66a9885a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:04.571956 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8bd8fa-5d5c-4ae0-99a6-09b0a5981aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:04.572825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bd8fa-5d5c-4ae0-99a6-09b0a5981aaa None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:04.573110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bd8fa-5d5c-4ae0-99a6-09b0a5981aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:04.573390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8bd8fa-5d5c-4ae0-99a6-09b0a5981aaa None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:04.574568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bd8fa-5d5c-4ae0-99a6-09b0a5981aaa None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:04.576104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8bd8fa-5d5c-4ae0-99a6-09b0a5981aaa None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:04.583148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 108/207] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:04 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:04.584687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b28b2-fb99-47a6-9157-bc1ff8222351 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:04.714706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b28b2-fb99-47a6-9157-bc1ff8222351 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:04.748487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083295c6-11d4-4096-b1ea-d41c2a5b6c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:04.749139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083295c6-11d4-4096-b1ea-d41c2a5b6c42 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:04.749423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083295c6-11d4-4096-b1ea-d41c2a5b6c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:04.749696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083295c6-11d4-4096-b1ea-d41c2a5b6c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:04.750529 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083295c6-11d4-4096-b1ea-d41c2a5b6c42 None admin] RBAC: Adding query filter params (role.id=dd4438dfc8ea468db528cc4cda58f3b8, scope.project.id=a2c6e66812894558944eb8f6f156896d, user.id=013c7ef73b0e4bc0b335f79c4f2a2ffa) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:04.750841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083295c6-11d4-4096-b1ea-d41c2a5b6c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:04.752316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083295c6-11d4-4096-b1ea-d41c2a5b6c42 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:04.758325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 100/208] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:04 2022] GET /identity/v3/role_assignments?user.id=013c7ef73b0e4bc0b335f79c4f2a2ffa&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=dd4438dfc8ea468db528cc4cda58f3b8 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:05.057345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8cdc0-ccc2-4bcd-a68f-282a129e9d52 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.057827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8cdc0-ccc2-4bcd-a68f-282a129e9d52 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.057827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8cdc0-ccc2-4bcd-a68f-282a129e9d52 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.059511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 109/209] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:05.066186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90529a8a-91f6-4db9-ba5a-2cb137461872 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.066696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90529a8a-91f6-4db9-ba5a-2cb137461872 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.066771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90529a8a-91f6-4db9-ba5a-2cb137461872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.084591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-90529a8a-91f6-4db9-ba5a-2cb137461872 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:05.085792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-90529a8a-91f6-4db9-ba5a-2cb137461872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:05.106610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:05.163171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 101/210] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:05.183119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8cdc0-ccc2-4bcd-a68f-282a129e9d52 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:05.186600 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:05.279003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb5013-677c-419d-9c52-cd1f8b24dce4 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.279321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb5013-677c-419d-9c52-cd1f8b24dce4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.279738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb5013-677c-419d-9c52-cd1f8b24dce4 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.281266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 102/211] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:05.286351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3260da-e03c-4db3-9a7c-dda0d2b280e1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.286896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3260da-e03c-4db3-9a7c-dda0d2b280e1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.287424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3260da-e03c-4db3-9a7c-dda0d2b280e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.301172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8cdc0-ccc2-4bcd-a68f-282a129e9d52 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:05.309952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ee3260da-e03c-4db3-9a7c-dda0d2b280e1 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:05.312109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ee3260da-e03c-4db3-9a7c-dda0d2b280e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:05.343207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec36abd-4d05-474b-9a88-2cdff414a3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:05.344383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec36abd-4d05-474b-9a88-2cdff414a3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.344957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec36abd-4d05-474b-9a88-2cdff414a3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.345440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec36abd-4d05-474b-9a88-2cdff414a3c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.357569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec36abd-4d05-474b-9a88-2cdff414a3c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-0ec36abd-4d05-474b-9a88-2cdff414a3c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:05.366904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:05.367831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:05.368621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:05.371429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 110/212] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:05.377119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec36abd-4d05-474b-9a88-2cdff414a3c1 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:05.403025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 103/213] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:05.428273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3260da-e03c-4db3-9a7c-dda0d2b280e1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:05.499534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:05.507599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec36abd-4d05-474b-9a88-2cdff414a3c1 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:05.513645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:05.537530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3260da-e03c-4db3-9a7c-dda0d2b280e1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:05.553581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ddeef-2c8b-4548-add9-d1d50a28b22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:05.554357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ddeef-2c8b-4548-add9-d1d50a28b22c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.554718 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ddeef-2c8b-4548-add9-d1d50a28b22c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.555091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ddeef-2c8b-4548-add9-d1d50a28b22c None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.555952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ddeef-2c8b-4548-add9-d1d50a28b22c None admin] RBAC: Adding query filter params (name=member) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:05.556351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ddeef-2c8b-4548-add9-d1d50a28b22c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:05.557814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ddeef-2c8b-4548-add9-d1d50a28b22c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:05.571383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 111/214] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:05.572322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ddeef-2c8b-4548-add9-d1d50a28b22c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:05.586412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe9457-1741-4bc0-adfb-8d1c3e297fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:05.587739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe9457-1741-4bc0-adfb-8d1c3e297fda None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.588223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe9457-1741-4bc0-adfb-8d1c3e297fda None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.588920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe9457-1741-4bc0-adfb-8d1c3e297fda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.590682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe9457-1741-4bc0-adfb-8d1c3e297fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:05.592960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe9457-1741-4bc0-adfb-8d1c3e297fda None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:05.595506 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 104/215] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:05.600854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe9457-1741-4bc0-adfb-8d1c3e297fda None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:05.701113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ddeef-2c8b-4548-add9-d1d50a28b22c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:05.732886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:05.735171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe9457-1741-4bc0-adfb-8d1c3e297fda None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:05.751176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27e72d-8747-4637-9772-502eedda7771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:05.752957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27e72d-8747-4637-9772-502eedda7771 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.753567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27e72d-8747-4637-9772-502eedda7771 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.754029 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27e72d-8747-4637-9772-502eedda7771 None admin] PATH_INFO: `/v3/users/e7fd7f420d9a4969806a613361b4ae33` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.773003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52017ee-8598-4fc7-a11b-cacf35b602da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:05.773483 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27e72d-8747-4637-9772-502eedda7771 None admin] RBAC: Authorizing `identity:get_user(user_id=e7fd7f420d9a4969806a613361b4ae33)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:05.773880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52017ee-8598-4fc7-a11b-cacf35b602da None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.774229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52017ee-8598-4fc7-a11b-cacf35b602da None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.774574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52017ee-8598-4fc7-a11b-cacf35b602da None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.775559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52017ee-8598-4fc7-a11b-cacf35b602da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:05.776057 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c27e72d-8747-4637-9772-502eedda7771 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:05.777201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52017ee-8598-4fc7-a11b-cacf35b602da None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:05.778187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 112/216] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:05 2022] GET /identity/v3/users/e7fd7f420d9a4969806a613361b4ae33 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:05.784156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27e72d-8747-4637-9772-502eedda7771 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:05.802348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 105/217] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:00:05 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:00:05.862178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb7e66-1bc1-4047-9501-35a2b9d8461b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.862482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb7e66-1bc1-4047-9501-35a2b9d8461b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.862674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb7e66-1bc1-4047-9501-35a2b9d8461b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.864571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 106/218] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:00:05.871197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7d2e9-692f-4b49-841e-30a8c5d6c14e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.871631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7d2e9-692f-4b49-841e-30a8c5d6c14e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.872068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7d2e9-692f-4b49-841e-30a8c5d6c14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.886328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:05.892843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8ed7d2e9-692f-4b49-841e-30a8c5d6c14e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:05.895289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8ed7d2e9-692f-4b49-841e-30a8c5d6c14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:05.921173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27e72d-8747-4637-9772-502eedda7771 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:05.959992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12cf4c5-d743-47e5-8e33-7b944bf8e6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:05.960991 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12cf4c5-d743-47e5-8e33-7b944bf8e6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:05.961430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12cf4c5-d743-47e5-8e33-7b944bf8e6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:05.961858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12cf4c5-d743-47e5-8e33-7b944bf8e6c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:05.976380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 107/219] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:06.000276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7d2e9-692f-4b49-841e-30a8c5d6c14e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:06.021726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:06.049265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12cf4c5-d743-47e5-8e33-7b944bf8e6c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:06.050747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12cf4c5-d743-47e5-8e33-7b944bf8e6c4 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:06.137273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7d2e9-692f-4b49-841e-30a8c5d6c14e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:06.173555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 113/220] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:00:06.178922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafbb3a-40f8-4c0a-9d5f-51a25888b570 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:06.178922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafbb3a-40f8-4c0a-9d5f-51a25888b570 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:06.178922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafbb3a-40f8-4c0a-9d5f-51a25888b570 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:06.178922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 114/221] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:06.181951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ee399-001f-4488-904e-5f32ed7e124b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:06.182218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafbb3a-40f8-4c0a-9d5f-51a25888b570 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:06.182780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ee399-001f-4488-904e-5f32ed7e124b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:06.183398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ee399-001f-4488-904e-5f32ed7e124b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:06.183398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ee399-001f-4488-904e-5f32ed7e124b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:06.189310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:06.202456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ee399-001f-4488-904e-5f32ed7e124b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ff9ee399-001f-4488-904e-5f32ed7e124b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:06.212277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:06.213217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 108/222] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:00:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:06.215857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2330f7-0b6d-4481-8014-6a24eca7b879 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:06.217408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2330f7-0b6d-4481-8014-6a24eca7b879 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:06.217408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2330f7-0b6d-4481-8014-6a24eca7b879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:06.240757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0f2330f7-0b6d-4481-8014-6a24eca7b879 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:06.242299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0f2330f7-0b6d-4481-8014-6a24eca7b879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:06.299014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:06.324673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafbb3a-40f8-4c0a-9d5f-51a25888b570 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:06.333535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 109/223] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:06.339527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2330f7-0b6d-4481-8014-6a24eca7b879 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:06.379826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bd30e-cca9-4f14-8355-2667f4e714ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:06.380967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bd30e-cca9-4f14-8355-2667f4e714ff None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:06.381254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bd30e-cca9-4f14-8355-2667f4e714ff None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:06.381532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bd30e-cca9-4f14-8355-2667f4e714ff None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:06.382898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bd30e-cca9-4f14-8355-2667f4e714ff None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:06.384973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bd30e-cca9-4f14-8355-2667f4e714ff None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:06.387155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 115/224] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:06 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:06.388925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bd30e-cca9-4f14-8355-2667f4e714ff None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:06.479159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2330f7-0b6d-4481-8014-6a24eca7b879 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:06.507936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bd30e-cca9-4f14-8355-2667f4e714ff None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:06.524492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:06.530450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afadbf8-3869-4d9d-ae53-fe083e628d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:06.531942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afadbf8-3869-4d9d-ae53-fe083e628d23 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:06.532464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afadbf8-3869-4d9d-ae53-fe083e628d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:06.533215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afadbf8-3869-4d9d-ae53-fe083e628d23 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:06.534431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afadbf8-3869-4d9d-ae53-fe083e628d23 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:06.535384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afadbf8-3869-4d9d-ae53-fe083e628d23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:06.537521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afadbf8-3869-4d9d-ae53-fe083e628d23 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:06.548214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfdcab-e723-4886-831f-bd7be2d5ea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:06.548998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfdcab-e723-4886-831f-bd7be2d5ea04 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:06.549282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfdcab-e723-4886-831f-bd7be2d5ea04 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:06.549560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfdcab-e723-4886-831f-bd7be2d5ea04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:06.551186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 110/225] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:06.552231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afadbf8-3869-4d9d-ae53-fe083e628d23 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:06.565342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfdcab-e723-4886-831f-bd7be2d5ea04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-eebfdcab-e723-4886-831f-bd7be2d5ea04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:06.573103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:06.573994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:06.574780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:06.579544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 116/226] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:00:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:06.580404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfdcab-e723-4886-831f-bd7be2d5ea04 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:06.594072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:06.676652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afadbf8-3869-4d9d-ae53-fe083e628d23 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:06.710292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfdcab-e723-4886-831f-bd7be2d5ea04 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:06.713853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531516a-2d28-4fbb-b091-5f6dc183dbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:06.714148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531516a-2d28-4fbb-b091-5f6dc183dbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:06.714434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531516a-2d28-4fbb-b091-5f6dc183dbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:06.714713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531516a-2d28-4fbb-b091-5f6dc183dbb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:06.715580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531516a-2d28-4fbb-b091-5f6dc183dbb9 None admin] RBAC: Adding query filter params (role.id=6e32448ba01f41bcafb4cb0a839691fe, scope.project.id=96f8d6ac74154c299166a6e2bf295a72, user.id=e7fd7f420d9a4969806a613361b4ae33) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:06.715898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531516a-2d28-4fbb-b091-5f6dc183dbb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:06.717903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531516a-2d28-4fbb-b091-5f6dc183dbb9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:06.724260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 111/227] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:06 2022] GET /identity/v3/role_assignments?user.id=e7fd7f420d9a4969806a613361b4ae33&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:06.725129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531516a-2d28-4fbb-b091-5f6dc183dbb9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:06.752634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e7e5c-cb65-4984-b66f-ff7444eefcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:06.753654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e7e5c-cb65-4984-b66f-ff7444eefcc1 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:06.753791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e7e5c-cb65-4984-b66f-ff7444eefcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:06.754072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258e7e5c-cb65-4984-b66f-ff7444eefcc1 None admin] PATH_INFO: `/v3/users/197283397a3e4623bbeab4274197d919` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:06.755415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e7e5c-cb65-4984-b66f-ff7444eefcc1 None admin] RBAC: Authorizing `identity:get_user(user_id=197283397a3e4623bbeab4274197d919)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:06.756908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258e7e5c-cb65-4984-b66f-ff7444eefcc1 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:06.758908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 117/228] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:06 2022] GET /identity/v3/users/197283397a3e4623bbeab4274197d919 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:06.764837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e7e5c-cb65-4984-b66f-ff7444eefcc1 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:06.797663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:06.853830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:06.873972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531516a-2d28-4fbb-b091-5f6dc183dbb9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:06.903523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258e7e5c-cb65-4984-b66f-ff7444eefcc1 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:06.947682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be345306-9949-4330-aab4-8f9b4f279545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:06.948295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be345306-9949-4330-aab4-8f9b4f279545 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:06.948565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be345306-9949-4330-aab4-8f9b4f279545 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:06.948839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be345306-9949-4330-aab4-8f9b4f279545 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:06.949604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be345306-9949-4330-aab4-8f9b4f279545 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:06.949911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be345306-9949-4330-aab4-8f9b4f279545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:06.953149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be345306-9949-4330-aab4-8f9b4f279545 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:06.967888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 112/229] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:06.973106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be345306-9949-4330-aab4-8f9b4f279545 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:06.973650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e3bc1-50d4-4afa-a81e-ecc2c288500f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:06.974767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e3bc1-50d4-4afa-a81e-ecc2c288500f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:06.975325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e3bc1-50d4-4afa-a81e-ecc2c288500f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:06.975803 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e3bc1-50d4-4afa-a81e-ecc2c288500f None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:06.977199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e3bc1-50d4-4afa-a81e-ecc2c288500f None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:06.978892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e3bc1-50d4-4afa-a81e-ecc2c288500f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:06.982026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 118/230] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:06 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:06.988817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e3bc1-50d4-4afa-a81e-ecc2c288500f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:07.117065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be345306-9949-4330-aab4-8f9b4f279545 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:07.132399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:07.154014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cd820-72a4-49c8-89dc-6175ae3cbec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:07.155450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd820-72a4-49c8-89dc-6175ae3cbec3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:07.155736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd820-72a4-49c8-89dc-6175ae3cbec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:07.156025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925cd820-72a4-49c8-89dc-6175ae3cbec3 None admin] PATH_INFO: `/v3/users/197283397a3e4623bbeab4274197d919` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:07.157249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cd820-72a4-49c8-89dc-6175ae3cbec3 None admin] RBAC: Authorizing `identity:get_user(user_id=197283397a3e4623bbeab4274197d919)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:07.158738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925cd820-72a4-49c8-89dc-6175ae3cbec3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:07.160433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 113/231] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:06 2022] GET /identity/v3/users/197283397a3e4623bbeab4274197d919 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:07.166058 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cd820-72a4-49c8-89dc-6175ae3cbec3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:07.225497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e3bc1-50d4-4afa-a81e-ecc2c288500f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:07.289336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b5e83-4ef9-4df9-9034-4304c110565d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:07.290341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b5e83-4ef9-4df9-9034-4304c110565d None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:07.290629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b5e83-4ef9-4df9-9034-4304c110565d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:07.290905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b5e83-4ef9-4df9-9034-4304c110565d None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72/users/197283397a3e4623bbeab4274197d919/roles/4301bced0259446cafb79118a53a015e` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:07.296897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b5e83-4ef9-4df9-9034-4304c110565d None admin] RBAC: Authorizing `identity:create_grant(project_id=96f8d6ac74154c299166a6e2bf295a72, user_id=197283397a3e4623bbeab4274197d919, role_id=4301bced0259446cafb79118a53a015e)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:07.298470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b5e83-4ef9-4df9-9034-4304c110565d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:07.307753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925cd820-72a4-49c8-89dc-6175ae3cbec3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:07.328018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 119/232] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:00:06 2022] PUT /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72/users/197283397a3e4623bbeab4274197d919/roles/4301bced0259446cafb79118a53a015e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:00:07.346411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9caa18-0d0d-474c-8513-c66fe61a9ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:07.347337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9caa18-0d0d-474c-8513-c66fe61a9ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:07.347627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9caa18-0d0d-474c-8513-c66fe61a9ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:07.347908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9caa18-0d0d-474c-8513-c66fe61a9ed4 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:07.349225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9caa18-0d0d-474c-8513-c66fe61a9ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:07.350709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9caa18-0d0d-474c-8513-c66fe61a9ed4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:07.352706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 114/233] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:07 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:07.358699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b5e83-4ef9-4df9-9034-4304c110565d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:07.375254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:07.511280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b5e83-4ef9-4df9-9034-4304c110565d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:07.547519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a951d-4de5-4c12-8d97-977fba8dd9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:07.548415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a951d-4de5-4c12-8d97-977fba8dd9a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:07.548699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a951d-4de5-4c12-8d97-977fba8dd9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:07.548973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a951d-4de5-4c12-8d97-977fba8dd9a4 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72/users/197283397a3e4623bbeab4274197d919/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:07.550944 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a951d-4de5-4c12-8d97-977fba8dd9a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=96f8d6ac74154c299166a6e2bf295a72, user_id=197283397a3e4623bbeab4274197d919, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:07.552498 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a951d-4de5-4c12-8d97-977fba8dd9a4 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:07.567248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 120/234] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:00:07 2022] PUT /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72/users/197283397a3e4623bbeab4274197d919/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:08.341857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87bcf1-c8e7-425c-b114-48ce26c30875 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:08.342428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87bcf1-c8e7-425c-b114-48ce26c30875 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:08.342513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87bcf1-c8e7-425c-b114-48ce26c30875 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:08.343896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 115/235] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:08.358578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dcadf-fa3c-402c-ade3-3bcd3d5da476 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:08.359020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dcadf-fa3c-402c-ade3-3bcd3d5da476 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:08.359306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dcadf-fa3c-402c-ade3-3bcd3d5da476 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:08.360735 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd087ddd-252a-43fb-bfd5-a78f5d382162 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:08.360925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 121/236] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:08.361424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd087ddd-252a-43fb-bfd5-a78f5d382162 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:08.361904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd087ddd-252a-43fb-bfd5-a78f5d382162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:08.382395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cd087ddd-252a-43fb-bfd5-a78f5d382162 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:08.383073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0500277-dcca-4926-8d32-60adb2100f81 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:08.383373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0500277-dcca-4926-8d32-60adb2100f81 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:08.383817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0500277-dcca-4926-8d32-60adb2100f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:08.384533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cd087ddd-252a-43fb-bfd5-a78f5d382162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:08.404980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-a0500277-dcca-4926-8d32-60adb2100f81 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:08.406175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a0500277-dcca-4926-8d32-60adb2100f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:08.457272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:08.494340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 116/237] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:08.503999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 122/238] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:08.504544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:08.517632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd087ddd-252a-43fb-bfd5-a78f5d382162 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:08.552553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0500277-dcca-4926-8d32-60adb2100f81 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:08.650825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd087ddd-252a-43fb-bfd5-a78f5d382162 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:08.688688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94096f-106e-40a9-ab7a-b18a1dedb238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:08.690063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94096f-106e-40a9-ab7a-b18a1dedb238 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:08.690552 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94096f-106e-40a9-ab7a-b18a1dedb238 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:08.691106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac94096f-106e-40a9-ab7a-b18a1dedb238 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:08.703322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac94096f-106e-40a9-ab7a-b18a1dedb238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ac94096f-106e-40a9-ab7a-b18a1dedb238 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:08.712318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:08.713198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:08.714014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:08.716728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 117/239] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:08.723664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94096f-106e-40a9-ab7a-b18a1dedb238 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:08.771456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:08.785129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0500277-dcca-4926-8d32-60adb2100f81 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:08.856215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac94096f-106e-40a9-ab7a-b18a1dedb238 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:08.862086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c3864-54e7-4d20-bd13-95f609ff0685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:08.862898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c3864-54e7-4d20-bd13-95f609ff0685 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:08.863219 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c3864-54e7-4d20-bd13-95f609ff0685 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:08.863506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c3864-54e7-4d20-bd13-95f609ff0685 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:08.869362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c3864-54e7-4d20-bd13-95f609ff0685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:08.870853 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c3864-54e7-4d20-bd13-95f609ff0685 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:08.872725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 123/240] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:08.878669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c3864-54e7-4d20-bd13-95f609ff0685 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:08.926287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2937ce-221e-4f02-8203-21b83d0ef74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:08.926956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2937ce-221e-4f02-8203-21b83d0ef74c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:08.927272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2937ce-221e-4f02-8203-21b83d0ef74c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:08.927554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2937ce-221e-4f02-8203-21b83d0ef74c None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:08.928325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2937ce-221e-4f02-8203-21b83d0ef74c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:08.928638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2937ce-221e-4f02-8203-21b83d0ef74c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:08.930026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2937ce-221e-4f02-8203-21b83d0ef74c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:08.950781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 118/241] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:08.973136 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2937ce-221e-4f02-8203-21b83d0ef74c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:09.002188 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:09.071579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c3864-54e7-4d20-bd13-95f609ff0685 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:09.113043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bab3b1-7f1e-4e8c-91da-f3d870aea5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:09.113726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bab3b1-7f1e-4e8c-91da-f3d870aea5f0 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:09.114018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bab3b1-7f1e-4e8c-91da-f3d870aea5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:09.114299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bab3b1-7f1e-4e8c-91da-f3d870aea5f0 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:09.115027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bab3b1-7f1e-4e8c-91da-f3d870aea5f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:09.116526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bab3b1-7f1e-4e8c-91da-f3d870aea5f0 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:09.140809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 124/242] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:00:08 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:00:09.156918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2937ce-221e-4f02-8203-21b83d0ef74c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:09.158599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:09.203198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2ccd7-20f5-4575-b962-c51f880b0a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:09.211066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ccd7-20f5-4575-b962-c51f880b0a04 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:09.211066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ccd7-20f5-4575-b962-c51f880b0a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:09.211066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2ccd7-20f5-4575-b962-c51f880b0a04 None admin] PATH_INFO: `/v3/users/df25309287ff4ebabf7b7462e55749e7` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:09.232105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2ccd7-20f5-4575-b962-c51f880b0a04 None admin] RBAC: Authorizing `identity:get_user(user_id=df25309287ff4ebabf7b7462e55749e7)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:09.234252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2ccd7-20f5-4575-b962-c51f880b0a04 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:09.236579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 119/243] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:08 2022] GET /identity/v3/users/df25309287ff4ebabf7b7462e55749e7 => generated 297 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:09.242759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd3ff5-5ac5-4033-9384-aef49fb8c2fd None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:09.243148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd3ff5-5ac5-4033-9384-aef49fb8c2fd None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:09.243441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd3ff5-5ac5-4033-9384-aef49fb8c2fd None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:09.244868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 125/244] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:09.245882 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2ccd7-20f5-4575-b962-c51f880b0a04 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:09.251688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc273-29ea-469e-97c9-24f5483b4f08 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:09.251972 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc273-29ea-469e-97c9-24f5483b4f08 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:09.252264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc273-29ea-469e-97c9-24f5483b4f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:09.280714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-cc6cc273-29ea-469e-97c9-24f5483b4f08 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:09.284891 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-cc6cc273-29ea-469e-97c9-24f5483b4f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:09.379370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 126/245] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:09.396791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2ccd7-20f5-4575-b962-c51f880b0a04 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:09.403470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cc273-29ea-469e-97c9-24f5483b4f08 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:09.444969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db091ede-a723-4aa6-be88-1a66eb3031a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:09.445775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db091ede-a723-4aa6-be88-1a66eb3031a8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:09.446061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db091ede-a723-4aa6-be88-1a66eb3031a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:09.446598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db091ede-a723-4aa6-be88-1a66eb3031a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:09.477423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:09.486313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:09.537170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db091ede-a723-4aa6-be88-1a66eb3031a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:09.538895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db091ede-a723-4aa6-be88-1a66eb3031a8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:09.550525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cc273-29ea-469e-97c9-24f5483b4f08 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:09.599641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cafc5-e6ae-4e49-833c-2685cd413868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:09.600447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cafc5-e6ae-4e49-833c-2685cd413868 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:09.600728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cafc5-e6ae-4e49-833c-2685cd413868 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:09.601001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cafc5-e6ae-4e49-833c-2685cd413868 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:09.621580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cafc5-e6ae-4e49-833c-2685cd413868 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-cf4cafc5-e6ae-4e49-833c-2685cd413868 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:09.633451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:09.634371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:09.635265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:09.637667 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 127/246] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:09.644103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cafc5-e6ae-4e49-833c-2685cd413868 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:09.670406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 120/247] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:09.675983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db091ede-a723-4aa6-be88-1a66eb3031a8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:09.745260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:09.776363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cafc5-e6ae-4e49-833c-2685cd413868 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:09.790416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:09.792784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db091ede-a723-4aa6-be88-1a66eb3031a8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:09.821239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd01c36-0548-4871-bbbb-f4020cd64a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:09.822386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd01c36-0548-4871-bbbb-f4020cd64a6e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:09.823000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd01c36-0548-4871-bbbb-f4020cd64a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:09.823510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd01c36-0548-4871-bbbb-f4020cd64a6e None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:09.824648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd01c36-0548-4871-bbbb-f4020cd64a6e None admin] RBAC: Adding query filter params (name=member) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:09.825158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd01c36-0548-4871-bbbb-f4020cd64a6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:09.827990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd01c36-0548-4871-bbbb-f4020cd64a6e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:09.839296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 128/248] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:09.840800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab388c40-4934-4d94-be2f-74becd9dc112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:09.841241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab388c40-4934-4d94-be2f-74becd9dc112 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:09.841592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab388c40-4934-4d94-be2f-74becd9dc112 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:09.842011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab388c40-4934-4d94-be2f-74becd9dc112 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:09.844121 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab388c40-4934-4d94-be2f-74becd9dc112 None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:09.845064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab388c40-4934-4d94-be2f-74becd9dc112 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:09.845372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd01c36-0548-4871-bbbb-f4020cd64a6e None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:09.847255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 121/249] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:09 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:09.853103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab388c40-4934-4d94-be2f-74becd9dc112 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:09.970743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd01c36-0548-4871-bbbb-f4020cd64a6e None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:09.998635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab388c40-4934-4d94-be2f-74becd9dc112 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:10.025390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de229bd9-3564-4afd-bdb0-d00a545ef26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:10.026268 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de229bd9-3564-4afd-bdb0-d00a545ef26e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.026549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de229bd9-3564-4afd-bdb0-d00a545ef26e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.026824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de229bd9-3564-4afd-bdb0-d00a545ef26e None admin] PATH_INFO: `/v3/users/e7fd7f420d9a4969806a613361b4ae33` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.030551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de229bd9-3564-4afd-bdb0-d00a545ef26e None admin] RBAC: Authorizing `identity:get_user(user_id=e7fd7f420d9a4969806a613361b4ae33)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:10.032043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de229bd9-3564-4afd-bdb0-d00a545ef26e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:10.038132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 129/250] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:09 2022] GET /identity/v3/users/e7fd7f420d9a4969806a613361b4ae33 => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:10.039542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972ff7c-8953-4c2a-8599-8c18ae8c8d3c None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.039833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972ff7c-8953-4c2a-8599-8c18ae8c8d3c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.040115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972ff7c-8953-4c2a-8599-8c18ae8c8d3c None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.042742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 130/251] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:10.043879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffa00a-68ab-47a4-8fd7-7a7d601c3e0c None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.044167 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffa00a-68ab-47a4-8fd7-7a7d601c3e0c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.044448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ffa00a-68ab-47a4-8fd7-7a7d601c3e0c None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.045777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:10.046208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 131/252] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:10.050009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455df97f-ba2b-4424-b8fb-321a95202187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:10.050884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffa00a-68ab-47a4-8fd7-7a7d601c3e0c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:10.051993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455df97f-ba2b-4424-b8fb-321a95202187 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.052652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455df97f-ba2b-4424-b8fb-321a95202187 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.052928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455df97f-ba2b-4424-b8fb-321a95202187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.053780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455df97f-ba2b-4424-b8fb-321a95202187 None admin] RBAC: Adding query filter params (role.id=dd4438dfc8ea468db528cc4cda58f3b8, scope.project.id=96f8d6ac74154c299166a6e2bf295a72, user.id=df25309287ff4ebabf7b7462e55749e7) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:10.054096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455df97f-ba2b-4424-b8fb-321a95202187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:10.055591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455df97f-ba2b-4424-b8fb-321a95202187 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:10.062089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 122/253] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:09 2022] GET /identity/v3/role_assignments?user.id=df25309287ff4ebabf7b7462e55749e7&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=dd4438dfc8ea468db528cc4cda58f3b8 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:10.063424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23489d-74d3-4aa6-831c-144c03ca3dda None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.063711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23489d-74d3-4aa6-831c-144c03ca3dda None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.063991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23489d-74d3-4aa6-831c-144c03ca3dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.081859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1c23489d-74d3-4aa6-831c-144c03ca3dda None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:10.083070 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1c23489d-74d3-4aa6-831c-144c03ca3dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:10.113073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:10.191987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ffa00a-68ab-47a4-8fd7-7a7d601c3e0c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:10.220922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 123/254] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:10.222264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d862607-3bdd-447c-b474-684bb82970e8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.222564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d862607-3bdd-447c-b474-684bb82970e8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.222843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d862607-3bdd-447c-b474-684bb82970e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.247892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e325c6b-8399-4e69-8b16-9a326b2c550a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:10.249364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e325c6b-8399-4e69-8b16-9a326b2c550a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.249595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e325c6b-8399-4e69-8b16-9a326b2c550a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.249872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e325c6b-8399-4e69-8b16-9a326b2c550a None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.253613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e325c6b-8399-4e69-8b16-9a326b2c550a None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:10.255156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e325c6b-8399-4e69-8b16-9a326b2c550a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:10.257143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 132/255] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:10 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:10.263319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e325c6b-8399-4e69-8b16-9a326b2c550a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:10.276342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7d862607-3bdd-447c-b474-684bb82970e8 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:10.277569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7d862607-3bdd-447c-b474-684bb82970e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:10.380977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e325c6b-8399-4e69-8b16-9a326b2c550a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:10.381507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 124/256] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:10.391242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d862607-3bdd-447c-b474-684bb82970e8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:10.398901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:10.439762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e4530-4e80-448a-9b8b-d2d8c680acde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:10.440666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4530-4e80-448a-9b8b-d2d8c680acde None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.440951 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4530-4e80-448a-9b8b-d2d8c680acde None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.441227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e4530-4e80-448a-9b8b-d2d8c680acde None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72/users/e7fd7f420d9a4969806a613361b4ae33/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.447816 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e4530-4e80-448a-9b8b-d2d8c680acde None admin] RBAC: Authorizing `identity:create_grant(project_id=96f8d6ac74154c299166a6e2bf295a72, user_id=e7fd7f420d9a4969806a613361b4ae33, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:10.449294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321e4530-4e80-448a-9b8b-d2d8c680acde None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:10.476516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:10.481470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 133/257] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:00:10 2022] PUT /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72/users/e7fd7f420d9a4969806a613361b4ae33/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:00:10.482226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e4530-4e80-448a-9b8b-d2d8c680acde None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:10.553404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d862607-3bdd-447c-b474-684bb82970e8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:10.626698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b810d3e-1d4e-4d5a-a035-d89f01487720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:10.627921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b810d3e-1d4e-4d5a-a035-d89f01487720 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.628422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b810d3e-1d4e-4d5a-a035-d89f01487720 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.628934 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b810d3e-1d4e-4d5a-a035-d89f01487720 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.652987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b810d3e-1d4e-4d5a-a035-d89f01487720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-6b810d3e-1d4e-4d5a-a035-d89f01487720 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:10.667074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:10.668062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:10.668858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 125/258] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:00:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:10.675648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b810d3e-1d4e-4d5a-a035-d89f01487720 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:10.753779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e4530-4e80-448a-9b8b-d2d8c680acde None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:10.823162 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37daf1f9-f300-492b-b5a3-3e905e3ebeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:10.823898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37daf1f9-f300-492b-b5a3-3e905e3ebeb7 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.824182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37daf1f9-f300-492b-b5a3-3e905e3ebeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.824456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37daf1f9-f300-492b-b5a3-3e905e3ebeb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.848098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b810d3e-1d4e-4d5a-a035-d89f01487720 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:10.853851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37daf1f9-f300-492b-b5a3-3e905e3ebeb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:10.862007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-37daf1f9-f300-492b-b5a3-3e905e3ebeb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:10.863798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:10.869098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:10.870069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 134/259] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:00:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:10.875194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37daf1f9-f300-492b-b5a3-3e905e3ebeb7 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:10.905260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717d4ac-99d2-4066-804b-81bf3422298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:10.905916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717d4ac-99d2-4066-804b-81bf3422298e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:10.906201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717d4ac-99d2-4066-804b-81bf3422298e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:10.906478 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717d4ac-99d2-4066-804b-81bf3422298e None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:10.907271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717d4ac-99d2-4066-804b-81bf3422298e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:10.907597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717d4ac-99d2-4066-804b-81bf3422298e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:10.908965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717d4ac-99d2-4066-804b-81bf3422298e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:10.915360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:10.930975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 126/260] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:10.938648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717d4ac-99d2-4066-804b-81bf3422298e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:11.043447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37daf1f9-f300-492b-b5a3-3e905e3ebeb7 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:11.087646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511dc63-34bf-4416-823e-39a41f405cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:11.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511dc63-34bf-4416-823e-39a41f405cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:11.088518 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511dc63-34bf-4416-823e-39a41f405cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:11.088789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511dc63-34bf-4416-823e-39a41f405cb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:11.089553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511dc63-34bf-4416-823e-39a41f405cb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:11.089859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511dc63-34bf-4416-823e-39a41f405cb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:11.091276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511dc63-34bf-4416-823e-39a41f405cb8 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:11.104145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 135/261] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:11.111795 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511dc63-34bf-4416-823e-39a41f405cb8 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:11.137592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717d4ac-99d2-4066-804b-81bf3422298e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:11.183383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f0611-261a-456f-bad6-413c9c3ca1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:11.184266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f0611-261a-456f-bad6-413c9c3ca1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:11.184607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f0611-261a-456f-bad6-413c9c3ca1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:11.184881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f0611-261a-456f-bad6-413c9c3ca1e7 None admin] PATH_INFO: `/v3/users/197283397a3e4623bbeab4274197d919` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:11.189498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f0611-261a-456f-bad6-413c9c3ca1e7 None admin] RBAC: Authorizing `identity:get_user(user_id=197283397a3e4623bbeab4274197d919)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:11.190957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f0611-261a-456f-bad6-413c9c3ca1e7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:11.192810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 127/262] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:10 2022] GET /identity/v3/users/197283397a3e4623bbeab4274197d919 => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:11.208385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f0611-261a-456f-bad6-413c9c3ca1e7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:11.286417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:11.325737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:11.350156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f0611-261a-456f-bad6-413c9c3ca1e7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:11.350778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511dc63-34bf-4416-823e-39a41f405cb8 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:11.394571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f061b5-639a-492a-b975-80942ef15c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:11.395620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f061b5-639a-492a-b975-80942ef15c93 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:11.396110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f061b5-639a-492a-b975-80942ef15c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:11.396586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f061b5-639a-492a-b975-80942ef15c93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:11.429064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106e112-eac3-4a13-a06a-500ec9457676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:11.429952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106e112-eac3-4a13-a06a-500ec9457676 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:11.430238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106e112-eac3-4a13-a06a-500ec9457676 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:11.430585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106e112-eac3-4a13-a06a-500ec9457676 None admin] PATH_INFO: `/v3/users/197283397a3e4623bbeab4274197d919` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:11.431873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106e112-eac3-4a13-a06a-500ec9457676 None admin] RBAC: Authorizing `identity:get_user(user_id=197283397a3e4623bbeab4274197d919)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:11.433165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106e112-eac3-4a13-a06a-500ec9457676 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:11.438986 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 136/263] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:11 2022] GET /identity/v3/users/197283397a3e4623bbeab4274197d919 => generated 283 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:11.440095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106e112-eac3-4a13-a06a-500ec9457676 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:11.476956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f061b5-639a-492a-b975-80942ef15c93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:11.480843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f061b5-639a-492a-b975-80942ef15c93 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:11.527515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:11.589656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 128/264] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:00:11.595874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f061b5-639a-492a-b975-80942ef15c93 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:11.702206 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f061b5-639a-492a-b975-80942ef15c93 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:11.738411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b7d01-7a55-4617-8d4f-255e5ef518e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:11.739274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106e112-eac3-4a13-a06a-500ec9457676 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:11.739888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b7d01-7a55-4617-8d4f-255e5ef518e8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:11.740370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b7d01-7a55-4617-8d4f-255e5ef518e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:11.740842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b7d01-7a55-4617-8d4f-255e5ef518e8 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:11.742779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b7d01-7a55-4617-8d4f-255e5ef518e8 None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:11.744489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b7d01-7a55-4617-8d4f-255e5ef518e8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:11.746483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 129/265] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:11 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:11.752540 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b7d01-7a55-4617-8d4f-255e5ef518e8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:11.787523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:11.803729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88a216-8827-45d8-aa55-2ecf5dc0c3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:11.804388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88a216-8827-45d8-aa55-2ecf5dc0c3ce None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:11.804682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88a216-8827-45d8-aa55-2ecf5dc0c3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:11.804959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88a216-8827-45d8-aa55-2ecf5dc0c3ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:11.839796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:11.883073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88a216-8827-45d8-aa55-2ecf5dc0c3ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:11.883073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88a216-8827-45d8-aa55-2ecf5dc0c3ce None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:11.887290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b7d01-7a55-4617-8d4f-255e5ef518e8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:11.949188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca9caf-8c05-4f8f-8086-52cc5d8e2f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:11.950204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca9caf-8c05-4f8f-8086-52cc5d8e2f6a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:11.950690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca9caf-8c05-4f8f-8086-52cc5d8e2f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:11.952086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efca9caf-8c05-4f8f-8086-52cc5d8e2f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:11.953222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca9caf-8c05-4f8f-8086-52cc5d8e2f6a None admin] RBAC: Adding query filter params (role.id=4301bced0259446cafb79118a53a015e, scope.project.id=96f8d6ac74154c299166a6e2bf295a72, user.id=197283397a3e4623bbeab4274197d919) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:11.953740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca9caf-8c05-4f8f-8086-52cc5d8e2f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:11.960762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efca9caf-8c05-4f8f-8086-52cc5d8e2f6a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:11.976064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 130/266] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:11 2022] GET /identity/v3/role_assignments?user.id=197283397a3e4623bbeab4274197d919&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=4301bced0259446cafb79118a53a015e => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:11.984025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 137/267] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:00:11.989334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca9caf-8c05-4f8f-8086-52cc5d8e2f6a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:12.105224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:12.132670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efca9caf-8c05-4f8f-8086-52cc5d8e2f6a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:12.168451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f5d79-e106-4fdf-82c0-a8dee3ec88bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:12.169674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f5d79-e106-4fdf-82c0-a8dee3ec88bc None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:12.170707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f5d79-e106-4fdf-82c0-a8dee3ec88bc None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:12.171216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f5d79-e106-4fdf-82c0-a8dee3ec88bc None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:12.172637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f5d79-e106-4fdf-82c0-a8dee3ec88bc None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:12.174287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f5d79-e106-4fdf-82c0-a8dee3ec88bc None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:12.176242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 131/268] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:11 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:12.182542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88a216-8827-45d8-aa55-2ecf5dc0c3ce None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:12.264157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:12.318519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88a216-8827-45d8-aa55-2ecf5dc0c3ce None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:12.359952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc364e1-6e53-4945-8f47-d23ecff6073f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:12.360618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc364e1-6e53-4945-8f47-d23ecff6073f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:12.360902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc364e1-6e53-4945-8f47-d23ecff6073f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:12.361177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc364e1-6e53-4945-8f47-d23ecff6073f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:12.362016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc364e1-6e53-4945-8f47-d23ecff6073f None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=96f8d6ac74154c299166a6e2bf295a72, user.id=197283397a3e4623bbeab4274197d919) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:12.362329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc364e1-6e53-4945-8f47-d23ecff6073f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:12.363806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc364e1-6e53-4945-8f47-d23ecff6073f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:12.371094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 138/269] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:12 2022] GET /identity/v3/role_assignments?user.id=197283397a3e4623bbeab4274197d919&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:12.573458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf4814-4c93-4121-b8ce-95fce915954b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:12.574392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf4814-4c93-4121-b8ce-95fce915954b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:12.574879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf4814-4c93-4121-b8ce-95fce915954b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:12.577577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 132/270] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:12.582790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c54e47-30ac-42a9-9acf-95d66dd5b8a5 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:12.583301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c54e47-30ac-42a9-9acf-95d66dd5b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:12.583371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c54e47-30ac-42a9-9acf-95d66dd5b8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:12.605504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c3c54e47-30ac-42a9-9acf-95d66dd5b8a5 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:12.606716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c3c54e47-30ac-42a9-9acf-95d66dd5b8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:12.654841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfab6ed-382a-4bd4-9619-28e3b7562665 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:12.655188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfab6ed-382a-4bd4-9619-28e3b7562665 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:12.655453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfab6ed-382a-4bd4-9619-28e3b7562665 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:12.657072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 133/271] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:12.662212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e7493-15dd-4145-9465-c89f9c33dd12 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:12.662500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e7493-15dd-4145-9465-c89f9c33dd12 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:12.662787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36e7493-15dd-4145-9465-c89f9c33dd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:12.684029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e36e7493-15dd-4145-9465-c89f9c33dd12 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:12.686420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e36e7493-15dd-4145-9465-c89f9c33dd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:12.692968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 139/272] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:12.718599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c54e47-30ac-42a9-9acf-95d66dd5b8a5 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:12.768748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 134/273] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:12.770435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709923de-1428-4811-ba42-e5ef704ff1b1 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:12.770879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709923de-1428-4811-ba42-e5ef704ff1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:12.771333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709923de-1428-4811-ba42-e5ef704ff1b1 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:12.772873 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 135/274] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:12.777941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef4a99-ea18-4d6f-91db-027ed53b7a8d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:12.778400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef4a99-ea18-4d6f-91db-027ed53b7a8d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:12.778824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef4a99-ea18-4d6f-91db-027ed53b7a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:12.794842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:12.799772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-06ef4a99-ea18-4d6f-91db-027ed53b7a8d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:12.801697 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-06ef4a99-ea18-4d6f-91db-027ed53b7a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:12.850420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c54e47-30ac-42a9-9acf-95d66dd5b8a5 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:12.876117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 136/275] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:12.876851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef4a99-ea18-4d6f-91db-027ed53b7a8d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:12.879032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:12.896403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3993c-8b3f-4c18-93be-7390e778268a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:12.897585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3993c-8b3f-4c18-93be-7390e778268a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:12.897879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3993c-8b3f-4c18-93be-7390e778268a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:12.898196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3993c-8b3f-4c18-93be-7390e778268a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:12.899413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3993c-8b3f-4c18-93be-7390e778268a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:12.901098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3993c-8b3f-4c18-93be-7390e778268a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:12.902988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 140/276] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:12.908270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3993c-8b3f-4c18-93be-7390e778268a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:12.992474 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef4a99-ea18-4d6f-91db-027ed53b7a8d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:13.015525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3993c-8b3f-4c18-93be-7390e778268a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:13.028832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c810d-693a-4f5c-a325-c88a222063a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:13.030183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c810d-693a-4f5c-a325-c88a222063a5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.030469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c810d-693a-4f5c-a325-c88a222063a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.030744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c810d-693a-4f5c-a325-c88a222063a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.038251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:13.041135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c810d-693a-4f5c-a325-c88a222063a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-013c810d-693a-4f5c-a325-c88a222063a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:13.048897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:13.049759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:13.050560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:13.054953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 137/277] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:13.055706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c810d-693a-4f5c-a325-c88a222063a5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:13.056665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8a4be-4d1e-4073-a820-273bb146964b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:13.057519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a4be-4d1e-4073-a820-273bb146964b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.057810 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a4be-4d1e-4073-a820-273bb146964b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.058122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8a4be-4d1e-4073-a820-273bb146964b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.071757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8a4be-4d1e-4073-a820-273bb146964b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-35c8a4be-4d1e-4073-a820-273bb146964b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:13.082203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:13.083100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:13.084205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 141/278] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:13.085411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8a4be-4d1e-4073-a820-273bb146964b None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:13.129277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:13.170830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c810d-693a-4f5c-a325-c88a222063a5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:13.192949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8a4be-4d1e-4073-a820-273bb146964b None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:13.212134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e46adf-e983-4454-832f-863be5636192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:13.212931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e46adf-e983-4454-832f-863be5636192 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.213222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e46adf-e983-4454-832f-863be5636192 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.213638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e46adf-e983-4454-832f-863be5636192 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.214342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e46adf-e983-4454-832f-863be5636192 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:13.215843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e46adf-e983-4454-832f-863be5636192 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:13.233557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea18f56-df9d-4a9c-b403-26e5b35efd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:13.234333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea18f56-df9d-4a9c-b403-26e5b35efd67 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.234639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea18f56-df9d-4a9c-b403-26e5b35efd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.234954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea18f56-df9d-4a9c-b403-26e5b35efd67 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.235759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea18f56-df9d-4a9c-b403-26e5b35efd67 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:13.236076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea18f56-df9d-4a9c-b403-26e5b35efd67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:13.237460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea18f56-df9d-4a9c-b403-26e5b35efd67 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:13.238852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 138/279] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:00:13 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:00:13.246546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 142/280] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:13.248744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e46adf-e983-4454-832f-863be5636192 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:13.252783 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea18f56-df9d-4a9c-b403-26e5b35efd67 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:13.303117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:13.371642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea18f56-df9d-4a9c-b403-26e5b35efd67 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:13.371877 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e46adf-e983-4454-832f-863be5636192 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:13.392142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:13.416121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79488ee-a62a-4246-99f3-6d05bb3dc61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:13.417570 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79488ee-a62a-4246-99f3-6d05bb3dc61d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.418855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8842e54-45e8-4fc2-9b37-4f320e46563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:13.419175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8842e54-45e8-4fc2-9b37-4f320e46563b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.419462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8842e54-45e8-4fc2-9b37-4f320e46563b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.419777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8842e54-45e8-4fc2-9b37-4f320e46563b None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.420977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8842e54-45e8-4fc2-9b37-4f320e46563b None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:13.420977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8842e54-45e8-4fc2-9b37-4f320e46563b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:13.422455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8842e54-45e8-4fc2-9b37-4f320e46563b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:13.423632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79488ee-a62a-4246-99f3-6d05bb3dc61d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.423824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79488ee-a62a-4246-99f3-6d05bb3dc61d None admin] PATH_INFO: `/v3/users/df25309287ff4ebabf7b7462e55749e7` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.425534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79488ee-a62a-4246-99f3-6d05bb3dc61d None admin] RBAC: Authorizing `identity:get_user(user_id=df25309287ff4ebabf7b7462e55749e7)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:13.427059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79488ee-a62a-4246-99f3-6d05bb3dc61d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:13.429247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 143/281] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:13 2022] GET /identity/v3/users/df25309287ff4ebabf7b7462e55749e7 => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:13.431633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 139/282] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:13.434562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79488ee-a62a-4246-99f3-6d05bb3dc61d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:13.437299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8842e54-45e8-4fc2-9b37-4f320e46563b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:13.542185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79488ee-a62a-4246-99f3-6d05bb3dc61d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:13.555836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:13.558574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8842e54-45e8-4fc2-9b37-4f320e46563b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:13.593030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe93b6-7fc9-4bdd-8012-5221e54b1ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:13.594608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe93b6-7fc9-4bdd-8012-5221e54b1ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.595101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe93b6-7fc9-4bdd-8012-5221e54b1ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.595556 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe93b6-7fc9-4bdd-8012-5221e54b1ff4 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.597009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe93b6-7fc9-4bdd-8012-5221e54b1ff4 None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:13.598645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe93b6-7fc9-4bdd-8012-5221e54b1ff4 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:13.600619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 144/283] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:13 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:13.604095 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6008cb9-0339-4f0d-8b27-b8406c934fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:13.605083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6008cb9-0339-4f0d-8b27-b8406c934fab None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.605443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6008cb9-0339-4f0d-8b27-b8406c934fab None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.605798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6008cb9-0339-4f0d-8b27-b8406c934fab None admin] PATH_INFO: `/v3/users/b40d839784054dcbb4d7f9475254b062` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.606633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe93b6-7fc9-4bdd-8012-5221e54b1ff4 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:13.621832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6008cb9-0339-4f0d-8b27-b8406c934fab None admin] RBAC: Authorizing `identity:get_user(user_id=b40d839784054dcbb4d7f9475254b062)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:13.623413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6008cb9-0339-4f0d-8b27-b8406c934fab None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:13.625445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 140/284] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:13 2022] GET /identity/v3/users/b40d839784054dcbb4d7f9475254b062 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:13.631228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6008cb9-0339-4f0d-8b27-b8406c934fab None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:13.685916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:13.744388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe93b6-7fc9-4bdd-8012-5221e54b1ff4 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:13.761075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6008cb9-0339-4f0d-8b27-b8406c934fab None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:13.786950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d69e6-3a7a-474e-90a6-5639340909c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:13.788545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d69e6-3a7a-474e-90a6-5639340909c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.789226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d69e6-3a7a-474e-90a6-5639340909c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.789654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d69e6-3a7a-474e-90a6-5639340909c2 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72/users/df25309287ff4ebabf7b7462e55749e7/roles/dd4438dfc8ea468db528cc4cda58f3b8` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.792586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d69e6-3a7a-474e-90a6-5639340909c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=96f8d6ac74154c299166a6e2bf295a72, user_id=df25309287ff4ebabf7b7462e55749e7, role_id=dd4438dfc8ea468db528cc4cda58f3b8)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:13.795242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d69e6-3a7a-474e-90a6-5639340909c2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:13.803751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7627bae-01f1-4ff3-84eb-3bae3ee1fd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:13.804593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7627bae-01f1-4ff3-84eb-3bae3ee1fd3a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.805036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7627bae-01f1-4ff3-84eb-3bae3ee1fd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.805404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7627bae-01f1-4ff3-84eb-3bae3ee1fd3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.806416 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7627bae-01f1-4ff3-84eb-3bae3ee1fd3a None admin] RBAC: Adding query filter params (role.id=6e32448ba01f41bcafb4cb0a839691fe, user.id=b40d839784054dcbb4d7f9475254b062, scope.system=all) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:13.806829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7627bae-01f1-4ff3-84eb-3bae3ee1fd3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:13.808893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7627bae-01f1-4ff3-84eb-3bae3ee1fd3a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:13.814375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 145/285] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:00:13 2022] PUT /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72/users/df25309287ff4ebabf7b7462e55749e7/roles/dd4438dfc8ea468db528cc4cda58f3b8 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:13.820803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 141/286] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jul 5 15:00:13 2022] GET /identity/v3/role_assignments?user.id=b40d839784054dcbb4d7f9475254b062&scope.system=all&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:13.821007 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c122b7-0d00-469b-9763-83e67a4e74ca None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.821007 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c122b7-0d00-469b-9763-83e67a4e74ca None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.821007 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c122b7-0d00-469b-9763-83e67a4e74ca None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.822031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 146/287] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:13.828705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8197e5-a587-462b-9c13-5e266012ae88 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:13.829033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8197e5-a587-462b-9c13-5e266012ae88 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:13.829288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8197e5-a587-462b-9c13-5e266012ae88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:13.850364 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2b8197e5-a587-462b-9c13-5e266012ae88 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:13.851692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2b8197e5-a587-462b-9c13-5e266012ae88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:13.941104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 142/288] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:13.981162 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c122b7-0d00-469b-9763-83e67a4e74ca None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:14.034463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:14.107990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c122b7-0d00-469b-9763-83e67a4e74ca None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:14.143143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f0bfd-1c7e-4cab-bd01-96c50532e0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:14.143956 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f0bfd-1c7e-4cab-bd01-96c50532e0bf None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:14.144304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f0bfd-1c7e-4cab-bd01-96c50532e0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:14.144588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f0bfd-1c7e-4cab-bd01-96c50532e0bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:14.156793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f0bfd-1c7e-4cab-bd01-96c50532e0bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-078f0bfd-1c7e-4cab-bd01-96c50532e0bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:14.164100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:14.164984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:14.165775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 147/289] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:14.171117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8197e5-a587-462b-9c13-5e266012ae88 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:14.203650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:14.275539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8197e5-a587-462b-9c13-5e266012ae88 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:14.310978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670143c2-0db2-42c8-b87b-ad3f723cb6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:14.311686 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670143c2-0db2-42c8-b87b-ad3f723cb6fa None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:14.311973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670143c2-0db2-42c8-b87b-ad3f723cb6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:14.312250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670143c2-0db2-42c8-b87b-ad3f723cb6fa None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:14.313008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670143c2-0db2-42c8-b87b-ad3f723cb6fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:14.313318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670143c2-0db2-42c8-b87b-ad3f723cb6fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:14.314720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670143c2-0db2-42c8-b87b-ad3f723cb6fa None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:14.323846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 143/290] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:14.332246 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f0bfd-1c7e-4cab-bd01-96c50532e0bf None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:14.460212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:14.464084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f0bfd-1c7e-4cab-bd01-96c50532e0bf None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:14.511312 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5d8e6-37e2-40f1-9c48-82f6cfacd405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:14.512195 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5d8e6-37e2-40f1-9c48-82f6cfacd405 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:14.512477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5d8e6-37e2-40f1-9c48-82f6cfacd405 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:14.512754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d5d8e6-37e2-40f1-9c48-82f6cfacd405 None admin] PATH_INFO: `/v3/users/e7fd7f420d9a4969806a613361b4ae33` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:14.514470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5d8e6-37e2-40f1-9c48-82f6cfacd405 None admin] RBAC: Authorizing `identity:get_user(user_id=e7fd7f420d9a4969806a613361b4ae33)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:14.515930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d5d8e6-37e2-40f1-9c48-82f6cfacd405 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:14.517721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 148/291] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:14 2022] GET /identity/v3/users/e7fd7f420d9a4969806a613361b4ae33 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:14.523207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670143c2-0db2-42c8-b87b-ad3f723cb6fa None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:14.619876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:14.624200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670143c2-0db2-42c8-b87b-ad3f723cb6fa None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:14.658903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2d641-0335-4d3b-8eee-fd0ef2273391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:14.659593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2d641-0335-4d3b-8eee-fd0ef2273391 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:14.659878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2d641-0335-4d3b-8eee-fd0ef2273391 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:14.660157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2d641-0335-4d3b-8eee-fd0ef2273391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:14.729427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2d641-0335-4d3b-8eee-fd0ef2273391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:14.730811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2d641-0335-4d3b-8eee-fd0ef2273391 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:14.822883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:14.830938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 144/292] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:14.840124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5d8e6-37e2-40f1-9c48-82f6cfacd405 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:14.975465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d5d8e6-37e2-40f1-9c48-82f6cfacd405 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:15.024378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd5e71d-5803-4528-a2c1-8b75544992ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:15.025213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e71d-5803-4528-a2c1-8b75544992ef None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:15.025499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e71d-5803-4528-a2c1-8b75544992ef None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:15.025777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5e71d-5803-4528-a2c1-8b75544992ef None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:15.026954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5e71d-5803-4528-a2c1-8b75544992ef None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:15.028505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd5e71d-5803-4528-a2c1-8b75544992ef None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:15.030310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 149/293] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:14 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:15.036242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2d641-0335-4d3b-8eee-fd0ef2273391 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:15.136958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2d641-0335-4d3b-8eee-fd0ef2273391 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:15.170974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222743bf-c4ab-4afb-ab2a-ad58e3af702b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:15.171653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222743bf-c4ab-4afb-ab2a-ad58e3af702b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:15.171937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222743bf-c4ab-4afb-ab2a-ad58e3af702b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:15.172215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222743bf-c4ab-4afb-ab2a-ad58e3af702b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:15.173049 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222743bf-c4ab-4afb-ab2a-ad58e3af702b None admin] RBAC: Adding query filter params (role.id=6e32448ba01f41bcafb4cb0a839691fe, scope.project.id=96f8d6ac74154c299166a6e2bf295a72, user.id=e7fd7f420d9a4969806a613361b4ae33) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:15.173359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222743bf-c4ab-4afb-ab2a-ad58e3af702b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:15.174929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222743bf-c4ab-4afb-ab2a-ad58e3af702b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:15.182441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 145/294] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:15 2022] GET /identity/v3/role_assignments?user.id=e7fd7f420d9a4969806a613361b4ae33&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:15.816649 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d70e31-6fe0-49a7-a752-12f2343f667f None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:15.817018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d70e31-6fe0-49a7-a752-12f2343f667f None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:15.817227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d70e31-6fe0-49a7-a752-12f2343f667f None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:15.818824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 150/295] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:15.824324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b056e0-d52a-45ee-a563-221902fb74f8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:15.824756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b056e0-d52a-45ee-a563-221902fb74f8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:15.824926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b056e0-d52a-45ee-a563-221902fb74f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:15.842439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-65b056e0-d52a-45ee-a563-221902fb74f8 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:15.843702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-65b056e0-d52a-45ee-a563-221902fb74f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:15.895963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:15.915173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 146/296] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:15.934526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d70e31-6fe0-49a7-a752-12f2343f667f None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:15.971634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:16.040554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d70e31-6fe0-49a7-a752-12f2343f667f None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:16.075357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f30192-78b7-48b5-a8a8-435656c08334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:16.076618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f30192-78b7-48b5-a8a8-435656c08334 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.077154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f30192-78b7-48b5-a8a8-435656c08334 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.077432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f30192-78b7-48b5-a8a8-435656c08334 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.088181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f30192-78b7-48b5-a8a8-435656c08334 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-f8f30192-78b7-48b5-a8a8-435656c08334 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:16.095964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:16.096867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:16.097671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 151/297] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:16.103342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b056e0-d52a-45ee-a563-221902fb74f8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:16.112621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c811d9-5133-43e3-8c23-e5dbde0de105 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.112988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c811d9-5133-43e3-8c23-e5dbde0de105 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.113283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c811d9-5133-43e3-8c23-e5dbde0de105 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.114906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 152/298] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:16.119769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3bee3-31e7-43cb-9571-2c4133ef5432 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.120066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3bee3-31e7-43cb-9571-2c4133ef5432 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.120407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b3bee3-31e7-43cb-9571-2c4133ef5432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.137844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-24b3bee3-31e7-43cb-9571-2c4133ef5432 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:16.139094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-24b3bee3-31e7-43cb-9571-2c4133ef5432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:16.213322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b056e0-d52a-45ee-a563-221902fb74f8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:16.217036 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 153/299] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:16.224014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd19fdc-e295-419c-bd92-77284c18707c None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.224870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd19fdc-e295-419c-bd92-77284c18707c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.225350 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd19fdc-e295-419c-bd92-77284c18707c None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.227569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 154/300] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:16 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:16.234154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8961f-431c-49c0-ac90-57e7c2dfa7e6 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.234456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8961f-431c-49c0-ac90-57e7c2dfa7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.234744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8961f-431c-49c0-ac90-57e7c2dfa7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.253230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0ed8961f-431c-49c0-ac90-57e7c2dfa7e6 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:16.254427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0ed8961f-431c-49c0-ac90-57e7c2dfa7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:16.254654 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470538f7-c630-4541-b8bd-1fac53c9641c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:16.255370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470538f7-c630-4541-b8bd-1fac53c9641c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.255658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470538f7-c630-4541-b8bd-1fac53c9641c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.255938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470538f7-c630-4541-b8bd-1fac53c9641c None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.256691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470538f7-c630-4541-b8bd-1fac53c9641c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:16.257005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470538f7-c630-4541-b8bd-1fac53c9641c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:16.258390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470538f7-c630-4541-b8bd-1fac53c9641c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:16.271588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 147/301] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:16.271588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470538f7-c630-4541-b8bd-1fac53c9641c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:16.276924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:16.321482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:16.332733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 155/302] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:16.333757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8961f-431c-49c0-ac90-57e7c2dfa7e6 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:16.382393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470538f7-c630-4541-b8bd-1fac53c9641c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:16.417253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bfdcd-2483-4d90-86bf-9289cde1210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:16.418135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bfdcd-2483-4d90-86bf-9289cde1210b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.418484 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bfdcd-2483-4d90-86bf-9289cde1210b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.418775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bfdcd-2483-4d90-86bf-9289cde1210b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.420048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bfdcd-2483-4d90-86bf-9289cde1210b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:16.421799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bfdcd-2483-4d90-86bf-9289cde1210b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:16.423459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 148/303] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:16.424220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bfdcd-2483-4d90-86bf-9289cde1210b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:16.444744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8961f-431c-49c0-ac90-57e7c2dfa7e6 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:16.481721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d9eb4-e9d9-4139-bb05-3a48463f1ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:16.483359 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d9eb4-e9d9-4139-bb05-3a48463f1ead None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.483790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d9eb4-e9d9-4139-bb05-3a48463f1ead None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.484256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d9eb4-e9d9-4139-bb05-3a48463f1ead None admin] PATH_INFO: `/v3/users/1af75e1b60414d0ab0abe2ad14b95495` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.497695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d9eb4-e9d9-4139-bb05-3a48463f1ead None admin] RBAC: Authorizing `identity:get_user(user_id=1af75e1b60414d0ab0abe2ad14b95495)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:16.499881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4d9eb4-e9d9-4139-bb05-3a48463f1ead None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:16.501716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 156/304] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:16 2022] GET /identity/v3/users/1af75e1b60414d0ab0abe2ad14b95495 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:16.502606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d9eb4-e9d9-4139-bb05-3a48463f1ead None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:16.520671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:16.533648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bfdcd-2483-4d90-86bf-9289cde1210b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:16.550467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:16.567265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7719fe-9573-4efe-a0b5-3b1be242be91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:16.568063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7719fe-9573-4efe-a0b5-3b1be242be91 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.568350 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7719fe-9573-4efe-a0b5-3b1be242be91 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.568628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7719fe-9573-4efe-a0b5-3b1be242be91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.578663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7719fe-9573-4efe-a0b5-3b1be242be91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0c7719fe-9573-4efe-a0b5-3b1be242be91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:16.586379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:16.587325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:16.588106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 149/305] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:16.588871 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7719fe-9573-4efe-a0b5-3b1be242be91 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:16.611752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d9eb4-e9d9-4139-bb05-3a48463f1ead None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:16.645459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc9fb7-0454-4599-840b-1ee09d4b4187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:16.646170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc9fb7-0454-4599-840b-1ee09d4b4187 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.646457 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc9fb7-0454-4599-840b-1ee09d4b4187 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.646739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc9fb7-0454-4599-840b-1ee09d4b4187 None admin] PATH_INFO: `/v3/groups` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.647463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc9fb7-0454-4599-840b-1ee09d4b4187 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:16.648903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc9fb7-0454-4599-840b-1ee09d4b4187 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:16.662422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 157/306] 10.222.0.23 () {60 vars in 1134 bytes} [Tue Jul 5 15:00:16 2022] POST /identity/v3/groups => generated 229 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:00:16.663774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a2920-ce35-4c69-9b9a-d6411125a882 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.664082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a2920-ce35-4c69-9b9a-d6411125a882 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.664363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a2920-ce35-4c69-9b9a-d6411125a882 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.665624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 158/307] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:16.666413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a2920-ce35-4c69-9b9a-d6411125a882 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:16.696126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7719fe-9573-4efe-a0b5-3b1be242be91 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:16.737777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a41ac-5972-439a-9cc7-fad9701b5802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:16.738457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a41ac-5972-439a-9cc7-fad9701b5802 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.738741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a41ac-5972-439a-9cc7-fad9701b5802 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.739020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a41ac-5972-439a-9cc7-fad9701b5802 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.739926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a41ac-5972-439a-9cc7-fad9701b5802 None admin] RBAC: Adding query filter params (role.id=dd4438dfc8ea468db528cc4cda58f3b8, user.id=1af75e1b60414d0ab0abe2ad14b95495, scope.system=all) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:16.740398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a41ac-5972-439a-9cc7-fad9701b5802 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:16.741870 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a41ac-5972-439a-9cc7-fad9701b5802 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:16.748286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 150/308] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jul 5 15:00:16 2022] GET /identity/v3/role_assignments?user.id=1af75e1b60414d0ab0abe2ad14b95495&scope.system=all&role.id=dd4438dfc8ea468db528cc4cda58f3b8 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:16.749677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9eeea-4e6e-4df1-a5d0-434e99b5649e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.749978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9eeea-4e6e-4df1-a5d0-434e99b5649e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.750262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9eeea-4e6e-4df1-a5d0-434e99b5649e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.759450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:16.769206 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5ca9eeea-4e6e-4df1-a5d0-434e99b5649e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:16.770564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5ca9eeea-4e6e-4df1-a5d0-434e99b5649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:16.811539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a2920-ce35-4c69-9b9a-d6411125a882 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:16.825839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:16.862783 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fccced-4d23-43ad-8d7d-6865ae861a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:16.863449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fccced-4d23-43ad-8d7d-6865ae861a17 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:16.863732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fccced-4d23-43ad-8d7d-6865ae861a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:16.864017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fccced-4d23-43ad-8d7d-6865ae861a17 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:16.864776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fccced-4d23-43ad-8d7d-6865ae861a17 None admin] RBAC: Adding query filter params (name=member) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:16.865084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fccced-4d23-43ad-8d7d-6865ae861a17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:16.866488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fccced-4d23-43ad-8d7d-6865ae861a17 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:16.869276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 151/309] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:16.877432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 159/310] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:16.883070 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9eeea-4e6e-4df1-a5d0-434e99b5649e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:16.896379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fccced-4d23-43ad-8d7d-6865ae861a17 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:16.999325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9eeea-4e6e-4df1-a5d0-434e99b5649e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:17.014721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fccced-4d23-43ad-8d7d-6865ae861a17 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:17.036014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde5525-1191-49a9-a500-80cc8ef7e955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:17.037010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde5525-1191-49a9-a500-80cc8ef7e955 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:17.037366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde5525-1191-49a9-a500-80cc8ef7e955 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:17.037715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde5525-1191-49a9-a500-80cc8ef7e955 None admin] PATH_INFO: `/v3/users/b40d839784054dcbb4d7f9475254b062` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:17.039333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde5525-1191-49a9-a500-80cc8ef7e955 None admin] RBAC: Authorizing `identity:get_user(user_id=b40d839784054dcbb4d7f9475254b062)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:17.040884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bde5525-1191-49a9-a500-80cc8ef7e955 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:17.042800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 152/311] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:16 2022] GET /identity/v3/users/b40d839784054dcbb4d7f9475254b062 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:17.048755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde5525-1191-49a9-a500-80cc8ef7e955 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:17.050925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dbb86-d541-4ad8-89b6-38ce25171cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:17.051835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dbb86-d541-4ad8-89b6-38ce25171cda None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:17.052193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dbb86-d541-4ad8-89b6-38ce25171cda None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:17.052546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dbb86-d541-4ad8-89b6-38ce25171cda None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:17.054636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:17.067207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188dbb86-d541-4ad8-89b6-38ce25171cda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-188dbb86-d541-4ad8-89b6-38ce25171cda None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:17.075466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:17.076876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:17.077878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:17.080249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 160/312] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:17.088334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dbb86-d541-4ad8-89b6-38ce25171cda None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:17.098641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:17.159574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde5525-1191-49a9-a500-80cc8ef7e955 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:17.205175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c9b96-c0da-4684-914b-b2a1fc9011f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:17.206062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c9b96-c0da-4684-914b-b2a1fc9011f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:17.206348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c9b96-c0da-4684-914b-b2a1fc9011f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:17.206631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c9b96-c0da-4684-914b-b2a1fc9011f3 None admin] PATH_INFO: `/v3/system/users/b40d839784054dcbb4d7f9475254b062/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:17.208219 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c9b96-c0da-4684-914b-b2a1fc9011f3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b40d839784054dcbb4d7f9475254b062, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:17.209574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882c9b96-c0da-4684-914b-b2a1fc9011f3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:17.218612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dbb86-d541-4ad8-89b6-38ce25171cda None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:17.227556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 153/313] 10.222.0.23 () {58 vars in 1334 bytes} [Tue Jul 5 15:00:17 2022] PUT /identity/v3/system/users/b40d839784054dcbb4d7f9475254b062/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:00:17.262236 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a0616-a6f7-4809-81f3-e8b3bac885d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:17.262980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a0616-a6f7-4809-81f3-e8b3bac885d9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:17.263874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a0616-a6f7-4809-81f3-e8b3bac885d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:17.264163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a0616-a6f7-4809-81f3-e8b3bac885d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:17.264947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a0616-a6f7-4809-81f3-e8b3bac885d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:17.265263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a0616-a6f7-4809-81f3-e8b3bac885d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:17.266835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a0616-a6f7-4809-81f3-e8b3bac885d9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:17.276932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 161/314] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:17.283075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c9b96-c0da-4684-914b-b2a1fc9011f3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:17.392287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c9b96-c0da-4684-914b-b2a1fc9011f3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:17.394645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:17.429406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734b5a5-a05d-47af-a638-03afcb89010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:17.430193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734b5a5-a05d-47af-a638-03afcb89010b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:17.430475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734b5a5-a05d-47af-a638-03afcb89010b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:17.430747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734b5a5-a05d-47af-a638-03afcb89010b None admin] PATH_INFO: `/v3/users/df25309287ff4ebabf7b7462e55749e7` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:17.431974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734b5a5-a05d-47af-a638-03afcb89010b None admin] RBAC: Authorizing `identity:get_user(user_id=df25309287ff4ebabf7b7462e55749e7)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:17.433412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734b5a5-a05d-47af-a638-03afcb89010b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:17.439418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 154/315] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:17 2022] GET /identity/v3/users/df25309287ff4ebabf7b7462e55749e7 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:17.440919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a0616-a6f7-4809-81f3-e8b3bac885d9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:17.499458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:17.545606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a0616-a6f7-4809-81f3-e8b3bac885d9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:17.579764 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14114e63-b3c6-4f3f-acc3-3f9c49144e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:17.580412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14114e63-b3c6-4f3f-acc3-3f9c49144e13 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:17.580696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14114e63-b3c6-4f3f-acc3-3f9c49144e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:17.580975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14114e63-b3c6-4f3f-acc3-3f9c49144e13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:17.651126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14114e63-b3c6-4f3f-acc3-3f9c49144e13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:17.652590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14114e63-b3c6-4f3f-acc3-3f9c49144e13 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:17.707480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:17.757766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 162/316] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:17.759181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734b5a5-a05d-47af-a638-03afcb89010b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:17.863978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734b5a5-a05d-47af-a638-03afcb89010b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:17.900700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b0d1c-1b7a-4646-af15-00488cc06670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:17.901574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b0d1c-1b7a-4646-af15-00488cc06670 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:17.901864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b0d1c-1b7a-4646-af15-00488cc06670 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:17.902132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b0d1c-1b7a-4646-af15-00488cc06670 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:17.903370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b0d1c-1b7a-4646-af15-00488cc06670 None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:17.905341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b0d1c-1b7a-4646-af15-00488cc06670 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:17.907425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 155/317] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:17 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:17.913321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14114e63-b3c6-4f3f-acc3-3f9c49144e13 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:18.018368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14114e63-b3c6-4f3f-acc3-3f9c49144e13 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:18.057085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fd4810-20ab-49d4-9b4f-a101cf6d6400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:18.057749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd4810-20ab-49d4-9b4f-a101cf6d6400 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:18.058038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd4810-20ab-49d4-9b4f-a101cf6d6400 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:18.058317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fd4810-20ab-49d4-9b4f-a101cf6d6400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:18.059183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd4810-20ab-49d4-9b4f-a101cf6d6400 None admin] RBAC: Adding query filter params (role.id=dd4438dfc8ea468db528cc4cda58f3b8, scope.project.id=96f8d6ac74154c299166a6e2bf295a72, user.id=df25309287ff4ebabf7b7462e55749e7) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:18.059500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd4810-20ab-49d4-9b4f-a101cf6d6400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:18.061001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fd4810-20ab-49d4-9b4f-a101cf6d6400 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:18.071152 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 163/318] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:17 2022] GET /identity/v3/role_assignments?user.id=df25309287ff4ebabf7b7462e55749e7&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=dd4438dfc8ea468db528cc4cda58f3b8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:18.755498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e2063-630e-4399-a491-4cbf4083cbb8 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:18.755896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e2063-630e-4399-a491-4cbf4083cbb8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:18.756069 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e2063-630e-4399-a491-4cbf4083cbb8 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:18.757601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 156/319] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:18.762405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886393f6-4105-41b9-aa40-7b161b35d14a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:18.762699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886393f6-4105-41b9-aa40-7b161b35d14a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:18.762982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886393f6-4105-41b9-aa40-7b161b35d14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:18.780322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-886393f6-4105-41b9-aa40-7b161b35d14a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:18.781517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-886393f6-4105-41b9-aa40-7b161b35d14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:18.796807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:18.816805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b9c6e-aab8-4929-a3e0-af1e87be0a0e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:18.817089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b9c6e-aab8-4929-a3e0-af1e87be0a0e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:18.817366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b9c6e-aab8-4929-a3e0-af1e87be0a0e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:18.818729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 157/320] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:18.823578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb1d7-905a-459b-946e-5c9f03df2dbd None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:18.823861 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb1d7-905a-459b-946e-5c9f03df2dbd None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:18.824201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb1d7-905a-459b-946e-5c9f03df2dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:18.840707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fa8cb1d7-905a-459b-946e-5c9f03df2dbd None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:18.841912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fa8cb1d7-905a-459b-946e-5c9f03df2dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:18.859867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 164/321] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:18.881518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:18.882485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886393f6-4105-41b9-aa40-7b161b35d14a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:18.922768 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 158/322] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:18.945268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cb1d7-905a-459b-946e-5c9f03df2dbd None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:18.999124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886393f6-4105-41b9-aa40-7b161b35d14a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:19.037062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53008b1-5d63-438d-89bb-b6434120a7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:19.038062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53008b1-5d63-438d-89bb-b6434120a7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:19.038566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53008b1-5d63-438d-89bb-b6434120a7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:19.039005 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53008b1-5d63-438d-89bb-b6434120a7d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:19.040883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53008b1-5d63-438d-89bb-b6434120a7d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:19.042816 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53008b1-5d63-438d-89bb-b6434120a7d2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:19.049962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 165/323] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:19.050124 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cb1d7-905a-459b-946e-5c9f03df2dbd None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:19.052091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53008b1-5d63-438d-89bb-b6434120a7d2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:19.070849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:19.088184 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdce1f-c91e-49e6-8ff4-60ccb3be0719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:19.089702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdce1f-c91e-49e6-8ff4-60ccb3be0719 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:19.089988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdce1f-c91e-49e6-8ff4-60ccb3be0719 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:19.090265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdce1f-c91e-49e6-8ff4-60ccb3be0719 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:19.102381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdce1f-c91e-49e6-8ff4-60ccb3be0719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0abdce1f-c91e-49e6-8ff4-60ccb3be0719 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:19.110569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:19.116680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:19.117494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 159/324] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:19.118159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdce1f-c91e-49e6-8ff4-60ccb3be0719 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:19.151828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:19.157078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53008b1-5d63-438d-89bb-b6434120a7d2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:19.202962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb21148-8aab-4bb6-8af7-ab0249f06edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:19.203723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb21148-8aab-4bb6-8af7-ab0249f06edd None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:19.204011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb21148-8aab-4bb6-8af7-ab0249f06edd None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:19.204333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb21148-8aab-4bb6-8af7-ab0249f06edd None admin] PATH_INFO: `/v3/groups` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:19.205440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb21148-8aab-4bb6-8af7-ab0249f06edd None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:19.207286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb21148-8aab-4bb6-8af7-ab0249f06edd None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:19.228213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 166/325] 10.222.0.23 () {60 vars in 1134 bytes} [Tue Jul 5 15:00:19 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:00:19.231778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdce1f-c91e-49e6-8ff4-60ccb3be0719 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:19.271336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ff418d-c640-471e-a35a-7c72aedf361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:19.272022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff418d-c640-471e-a35a-7c72aedf361f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:19.272308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff418d-c640-471e-a35a-7c72aedf361f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:19.272587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ff418d-c640-471e-a35a-7c72aedf361f None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:19.273441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff418d-c640-471e-a35a-7c72aedf361f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:19.273758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff418d-c640-471e-a35a-7c72aedf361f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:19.275309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ff418d-c640-471e-a35a-7c72aedf361f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:19.285650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 160/326] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:19.291286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb21148-8aab-4bb6-8af7-ab0249f06edd None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:19.367907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531cad-b45b-4261-a8ec-d92f74fab11c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:19.368203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531cad-b45b-4261-a8ec-d92f74fab11c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:19.368485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee531cad-b45b-4261-a8ec-d92f74fab11c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:19.370030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 161/327] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:00:19.375827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67d3d-d22d-4d15-a8de-88b97fa78090 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:19.376123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67d3d-d22d-4d15-a8de-88b97fa78090 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:19.376416 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b67d3d-d22d-4d15-a8de-88b97fa78090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:19.377621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:19.394131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d1b67d3d-d22d-4d15-a8de-88b97fa78090 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:19.395385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d1b67d3d-d22d-4d15-a8de-88b97fa78090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:19.404670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb21148-8aab-4bb6-8af7-ab0249f06edd None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:19.440629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1af72b-b296-42d7-8497-5a33f3945ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:19.441504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af72b-b296-42d7-8497-5a33f3945ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:19.442433 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af72b-b296-42d7-8497-5a33f3945ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:19.442433 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1af72b-b296-42d7-8497-5a33f3945ad2 None admin] PATH_INFO: `/v3/users/1af75e1b60414d0ab0abe2ad14b95495` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:19.443878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1af72b-b296-42d7-8497-5a33f3945ad2 None admin] RBAC: Authorizing `identity:get_user(user_id=1af75e1b60414d0ab0abe2ad14b95495)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:19.445457 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1af72b-b296-42d7-8497-5a33f3945ad2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:19.447418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 167/328] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:19 2022] GET /identity/v3/users/1af75e1b60414d0ab0abe2ad14b95495 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:19.453200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1af72b-b296-42d7-8497-5a33f3945ad2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:19.479514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 162/329] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:19.504465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b67d3d-d22d-4d15-a8de-88b97fa78090 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:19.532409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:19.585464 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1af72b-b296-42d7-8497-5a33f3945ad2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:19.624301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b67d3d-d22d-4d15-a8de-88b97fa78090 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:19.634599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab170e7a-2a93-4061-85d5-07c2f48281fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:19.637738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab170e7a-2a93-4061-85d5-07c2f48281fc None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:19.637738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab170e7a-2a93-4061-85d5-07c2f48281fc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:19.637738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab170e7a-2a93-4061-85d5-07c2f48281fc None admin] PATH_INFO: `/v3/system/users/1af75e1b60414d0ab0abe2ad14b95495/roles/dd4438dfc8ea468db528cc4cda58f3b8` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:19.640624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab170e7a-2a93-4061-85d5-07c2f48281fc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1af75e1b60414d0ab0abe2ad14b95495, role_id=dd4438dfc8ea468db528cc4cda58f3b8)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:19.641918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab170e7a-2a93-4061-85d5-07c2f48281fc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:19.656866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 168/330] 10.222.0.23 () {58 vars in 1334 bytes} [Tue Jul 5 15:00:19 2022] PUT /identity/v3/system/users/1af75e1b60414d0ab0abe2ad14b95495/roles/dd4438dfc8ea468db528cc4cda58f3b8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:19.667371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02078c45-490c-4bac-b34b-c37da7a98ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:19.668181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02078c45-490c-4bac-b34b-c37da7a98ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:19.668466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02078c45-490c-4bac-b34b-c37da7a98ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:19.668746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02078c45-490c-4bac-b34b-c37da7a98ea7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:19.681888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02078c45-490c-4bac-b34b-c37da7a98ea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-02078c45-490c-4bac-b34b-c37da7a98ea7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:19.689895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:19.690863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:19.691951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 163/331] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:19.700277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab170e7a-2a93-4061-85d5-07c2f48281fc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:19.715164 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:19.818177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab170e7a-2a93-4061-85d5-07c2f48281fc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:19.858135 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28626e4e-a628-4b97-80c4-5dc8ef3dbb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:19.858736 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28626e4e-a628-4b97-80c4-5dc8ef3dbb86 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:19.859016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28626e4e-a628-4b97-80c4-5dc8ef3dbb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:19.859319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28626e4e-a628-4b97-80c4-5dc8ef3dbb86 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:19.860081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28626e4e-a628-4b97-80c4-5dc8ef3dbb86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:19.860388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28626e4e-a628-4b97-80c4-5dc8ef3dbb86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:19.861819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28626e4e-a628-4b97-80c4-5dc8ef3dbb86 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:19.871379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 169/332] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:19.877229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02078c45-490c-4bac-b34b-c37da7a98ea7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:19.969410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:19.989288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02078c45-490c-4bac-b34b-c37da7a98ea7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:20.026272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a51874-6400-4006-ba9d-45d01149e8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:20.027194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a51874-6400-4006-ba9d-45d01149e8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:20.027534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a51874-6400-4006-ba9d-45d01149e8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:20.027816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a51874-6400-4006-ba9d-45d01149e8a3 None admin] PATH_INFO: `/v3/users/b40d839784054dcbb4d7f9475254b062` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:20.029657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a51874-6400-4006-ba9d-45d01149e8a3 None admin] RBAC: Authorizing `identity:get_user(user_id=b40d839784054dcbb4d7f9475254b062)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:20.031176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a51874-6400-4006-ba9d-45d01149e8a3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:20.032989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 164/333] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:19 2022] GET /identity/v3/users/b40d839784054dcbb4d7f9475254b062 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:20.038603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28626e4e-a628-4b97-80c4-5dc8ef3dbb86 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:20.126528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:20.166937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28626e4e-a628-4b97-80c4-5dc8ef3dbb86 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:20.222066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56533e60-fa24-4db2-81f1-af5afd26c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:20.223091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56533e60-fa24-4db2-81f1-af5afd26c68f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:20.223607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56533e60-fa24-4db2-81f1-af5afd26c68f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:20.224086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56533e60-fa24-4db2-81f1-af5afd26c68f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:20.225121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56533e60-fa24-4db2-81f1-af5afd26c68f None admin] RBAC: Adding query filter params (role.id=6e32448ba01f41bcafb4cb0a839691fe, user.id=b40d839784054dcbb4d7f9475254b062, scope.system=all) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:20.225654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56533e60-fa24-4db2-81f1-af5afd26c68f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:20.227321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56533e60-fa24-4db2-81f1-af5afd26c68f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:20.240452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 170/334] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jul 5 15:00:20 2022] GET /identity/v3/role_assignments?user.id=b40d839784054dcbb4d7f9475254b062&scope.system=all&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 510 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:21.725924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490567af-019a-41ef-a63e-7f4ddeb9a5b3 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:21.726430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490567af-019a-41ef-a63e-7f4ddeb9a5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:21.726694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490567af-019a-41ef-a63e-7f4ddeb9a5b3 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:21.728214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 165/335] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:21.733558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cba39a-5a8c-4c58-940e-34362236e5d2 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:21.733981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cba39a-5a8c-4c58-940e-34362236e5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:21.734155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cba39a-5a8c-4c58-940e-34362236e5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:21.753607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-04cba39a-5a8c-4c58-940e-34362236e5d2 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:21.754799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-04cba39a-5a8c-4c58-940e-34362236e5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:21.834874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 171/336] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:21.868654 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490567af-019a-41ef-a63e-7f4ddeb9a5b3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:21.990949 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490567af-019a-41ef-a63e-7f4ddeb9a5b3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:22.030371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251572b-c77f-4d11-af90-3bd310931246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:22.031200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251572b-c77f-4d11-af90-3bd310931246 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.031487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251572b-c77f-4d11-af90-3bd310931246 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.031762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251572b-c77f-4d11-af90-3bd310931246 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.048858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251572b-c77f-4d11-af90-3bd310931246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-3251572b-c77f-4d11-af90-3bd310931246 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:22.056626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:22.057538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:22.058333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 166/337] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:22.065753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cba39a-5a8c-4c58-940e-34362236e5d2 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:22.140940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:22.194171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cba39a-5a8c-4c58-940e-34362236e5d2 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:22.239008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786258bb-269b-4fd9-a9d7-557984986b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:22.239758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786258bb-269b-4fd9-a9d7-557984986b08 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.240043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786258bb-269b-4fd9-a9d7-557984986b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.240317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786258bb-269b-4fd9-a9d7-557984986b08 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.241141 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786258bb-269b-4fd9-a9d7-557984986b08 None admin] RBAC: Adding query filter params (name=member) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:22.241426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786258bb-269b-4fd9-a9d7-557984986b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:22.242872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786258bb-269b-4fd9-a9d7-557984986b08 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:22.254849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 172/338] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:22.260324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251572b-c77f-4d11-af90-3bd310931246 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:22.275420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:22.383797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251572b-c77f-4d11-af90-3bd310931246 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:22.425558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d22258-1238-4260-8551-23a97340c070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:22.426219 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d22258-1238-4260-8551-23a97340c070 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.426505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d22258-1238-4260-8551-23a97340c070 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.426779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d22258-1238-4260-8551-23a97340c070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.454867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ce997-2cfc-4a79-b99e-4e86df1c5d91 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.455211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ce997-2cfc-4a79-b99e-4e86df1c5d91 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.455459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ce997-2cfc-4a79-b99e-4e86df1c5d91 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.457073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 173/339] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:00:22.463233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59339e-76a3-4026-9e9e-a8fa3d5e3ee3 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.463837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59339e-76a3-4026-9e9e-a8fa3d5e3ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.464364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a59339e-76a3-4026-9e9e-a8fa3d5e3ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.496124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0a59339e-76a3-4026-9e9e-a8fa3d5e3ee3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:22.497818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0a59339e-76a3-4026-9e9e-a8fa3d5e3ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:22.529510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d22258-1238-4260-8551-23a97340c070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:22.531070 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d22258-1238-4260-8551-23a97340c070 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:22.550659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:22.618768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 174/340] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:22.620457 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcd56f-d8ea-4b6d-81d4-c7f5a1f81b99 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.624185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcd56f-d8ea-4b6d-81d4-c7f5a1f81b99 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.624680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcd56f-d8ea-4b6d-81d4-c7f5a1f81b99 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.626205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 175/341] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:22.631562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aaa24-d150-4247-806a-a6d5440d7f6c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.632094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aaa24-d150-4247-806a-a6d5440d7f6c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.632577 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030aaa24-d150-4247-806a-a6d5440d7f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.661313 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-030aaa24-d150-4247-806a-a6d5440d7f6c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:22.662837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-030aaa24-d150-4247-806a-a6d5440d7f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:22.697590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 167/342] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:22.699640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d22258-1238-4260-8551-23a97340c070 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:22.756206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:22.770176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 176/343] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:22.771053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030aaa24-d150-4247-806a-a6d5440d7f6c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:22.813813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d22258-1238-4260-8551-23a97340c070 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:22.826223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:22.855192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b951eace-edba-4f4d-8f67-3a898caf9429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:22.856391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951eace-edba-4f4d-8f67-3a898caf9429 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.857417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951eace-edba-4f4d-8f67-3a898caf9429 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.857898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b951eace-edba-4f4d-8f67-3a898caf9429 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.871524 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b951eace-edba-4f4d-8f67-3a898caf9429 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b951eace-edba-4f4d-8f67-3a898caf9429 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:22.880343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:22.881267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 05 15:00:22.882126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:22.885145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 168/344] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:22.886674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d44c2-6914-4aca-ad01-d93b217470cb None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.887297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d44c2-6914-4aca-ad01-d93b217470cb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.887782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d44c2-6914-4aca-ad01-d93b217470cb None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.889679 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 169/345] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:22.890666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d44c2-6914-4aca-ad01-d93b217470cb None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:22.894220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030aaa24-d150-4247-806a-a6d5440d7f6c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:22.943417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3e42e-b432-4774-96de-124c418e51e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:22.944072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3e42e-b432-4774-96de-124c418e51e5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.944448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3e42e-b432-4774-96de-124c418e51e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.944734 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3e42e-b432-4774-96de-124c418e51e5 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.945982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3e42e-b432-4774-96de-124c418e51e5 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:22.947521 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3e42e-b432-4774-96de-124c418e51e5 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:22.949297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 177/346] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:22 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:22.950597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13695456-22e5-4b27-9bb5-b09067f5e243 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:22.950894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13695456-22e5-4b27-9bb5-b09067f5e243 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:22.954008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13695456-22e5-4b27-9bb5-b09067f5e243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:22.975874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-13695456-22e5-4b27-9bb5-b09067f5e243 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:22.977053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-13695456-22e5-4b27-9bb5-b09067f5e243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:23.028458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:23.037466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d44c2-6914-4aca-ad01-d93b217470cb None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:23.054838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 178/347] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:23.056329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13695456-22e5-4b27-9bb5-b09067f5e243 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:23.084778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f7a36-1282-4310-a23e-54305e0d48af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:23.086412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f7a36-1282-4310-a23e-54305e0d48af None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:23.087169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f7a36-1282-4310-a23e-54305e0d48af None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:23.087633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728f7a36-1282-4310-a23e-54305e0d48af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:23.103403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728f7a36-1282-4310-a23e-54305e0d48af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-728f7a36-1282-4310-a23e-54305e0d48af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:23.113321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:23.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:23.115181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:23.118694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 170/348] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:23.119836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f7a36-1282-4310-a23e-54305e0d48af None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:23.162818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:23.186909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13695456-22e5-4b27-9bb5-b09067f5e243 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:23.234657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7d180-255e-4895-875e-f6477d209c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:23.235491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d180-255e-4895-875e-f6477d209c37 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:23.235848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d180-255e-4895-875e-f6477d209c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:23.236204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7d180-255e-4895-875e-f6477d209c37 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:23.237331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7d180-255e-4895-875e-f6477d209c37 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:23.237724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7d180-255e-4895-875e-f6477d209c37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:23.239905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7d180-255e-4895-875e-f6477d209c37 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:23.251125 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728f7a36-1282-4310-a23e-54305e0d48af None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:23.251383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 179/349] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:23.252437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7d180-255e-4895-875e-f6477d209c37 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:23.293234 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520772-30c7-4c9a-9e25-cc36d31e6465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:23.294688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520772-30c7-4c9a-9e25-cc36d31e6465 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:23.295894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520772-30c7-4c9a-9e25-cc36d31e6465 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:23.296375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520772-30c7-4c9a-9e25-cc36d31e6465 None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:23.302786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f520772-30c7-4c9a-9e25-cc36d31e6465 None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:23.304538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f520772-30c7-4c9a-9e25-cc36d31e6465 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:23.306442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 171/350] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:23 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:23.307423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520772-30c7-4c9a-9e25-cc36d31e6465 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:23.308969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:23.389831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7d180-255e-4895-875e-f6477d209c37 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:23.416119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:23.426004 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520772-30c7-4c9a-9e25-cc36d31e6465 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:23.441159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f8bee-209c-4386-aa5b-58c4eb98d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:23.441987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f8bee-209c-4386-aa5b-58c4eb98d53b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:23.442277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f8bee-209c-4386-aa5b-58c4eb98d53b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:23.442569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f8bee-209c-4386-aa5b-58c4eb98d53b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:23.458407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f8bee-209c-4386-aa5b-58c4eb98d53b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:23.468881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493d9ff-3d52-4ff8-af20-4bd206516cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-422f8bee-209c-4386-aa5b-58c4eb98d53b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:23.469059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:23.469930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:23.470721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 180/351] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:00:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:23.471456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f8bee-209c-4386-aa5b-58c4eb98d53b None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:23.472967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493d9ff-3d52-4ff8-af20-4bd206516cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:23.473510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493d9ff-3d52-4ff8-af20-4bd206516cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:23.473986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493d9ff-3d52-4ff8-af20-4bd206516cb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:23.475028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493d9ff-3d52-4ff8-af20-4bd206516cb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:23.475623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493d9ff-3d52-4ff8-af20-4bd206516cb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:23.477335 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493d9ff-3d52-4ff8-af20-4bd206516cb0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:23.487629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 172/352] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:23.488729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493d9ff-3d52-4ff8-af20-4bd206516cb0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:23.600137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f8bee-209c-4386-aa5b-58c4eb98d53b None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:23.602222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:23.630393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493d9ff-3d52-4ff8-af20-4bd206516cb0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:23.647345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f9d66-19c0-4b7d-b9fb-ce808d1b4be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:23.648392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f9d66-19c0-4b7d-b9fb-ce808d1b4be8 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:23.648775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f9d66-19c0-4b7d-b9fb-ce808d1b4be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:23.649149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f9d66-19c0-4b7d-b9fb-ce808d1b4be8 None admin] PATH_INFO: `/v3/users/1af75e1b60414d0ab0abe2ad14b95495` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:23.650447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f9d66-19c0-4b7d-b9fb-ce808d1b4be8 None admin] RBAC: Authorizing `identity:get_user(user_id=1af75e1b60414d0ab0abe2ad14b95495)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:23.652061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f9d66-19c0-4b7d-b9fb-ce808d1b4be8 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:23.654519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 181/353] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:00:23 2022] GET /identity/v3/users/1af75e1b60414d0ab0abe2ad14b95495 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:23.656304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f9d66-19c0-4b7d-b9fb-ce808d1b4be8 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:23.669001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1aca2d-3d3a-4478-a171-d4e682bc235c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:23.669655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aca2d-3d3a-4478-a171-d4e682bc235c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:23.669947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aca2d-3d3a-4478-a171-d4e682bc235c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:23.670231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aca2d-3d3a-4478-a171-d4e682bc235c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:23.672034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1aca2d-3d3a-4478-a171-d4e682bc235c None admin] RBAC: Adding query filter params (group.id=670d33c6c7f24ab1b3c3e3880dae2beb, role.id=6e32448ba01f41bcafb4cb0a839691fe, scope.project.id=a2c6e66812894558944eb8f6f156896d) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:23.672034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1aca2d-3d3a-4478-a171-d4e682bc235c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:23.672939 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1aca2d-3d3a-4478-a171-d4e682bc235c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:23.683260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 173/354] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 5 15:00:23 2022] GET /identity/v3/role_assignments?group.id=670d33c6c7f24ab1b3c3e3880dae2beb&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:23.683260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1aca2d-3d3a-4478-a171-d4e682bc235c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:23.720070 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:23.800867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f9d66-19c0-4b7d-b9fb-ce808d1b4be8 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:23.839301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1aca2d-3d3a-4478-a171-d4e682bc235c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:23.848616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b2692f-782e-4d3c-9618-6161a9eac174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:23.849300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2692f-782e-4d3c-9618-6161a9eac174 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:23.849585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2692f-782e-4d3c-9618-6161a9eac174 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:23.849860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2692f-782e-4d3c-9618-6161a9eac174 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:23.850709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2692f-782e-4d3c-9618-6161a9eac174 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:23.851581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2692f-782e-4d3c-9618-6161a9eac174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:23.853181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b2692f-782e-4d3c-9618-6161a9eac174 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:23.863770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 182/355] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:23.865122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e10c-bd73-43df-a403-02623f9507a5 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:23.865423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e10c-bd73-43df-a403-02623f9507a5 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:23.865709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045e10c-bd73-43df-a403-02623f9507a5 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:23.867101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 183/356] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:23.867708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045e10c-bd73-43df-a403-02623f9507a5 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:23.886217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391018f6-453a-4ab3-a9ce-899762bd1240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:23.887821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391018f6-453a-4ab3-a9ce-899762bd1240 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:23.888941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391018f6-453a-4ab3-a9ce-899762bd1240 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:23.889372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391018f6-453a-4ab3-a9ce-899762bd1240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:23.905674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:23.975600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391018f6-453a-4ab3-a9ce-899762bd1240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:23.977450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391018f6-453a-4ab3-a9ce-899762bd1240 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:24.000156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045e10c-bd73-43df-a403-02623f9507a5 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:24.002801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:24.043759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f08264-932a-426c-abbe-456a033cbb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:24.044469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f08264-932a-426c-abbe-456a033cbb13 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:24.044761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f08264-932a-426c-abbe-456a033cbb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:24.045043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f08264-932a-426c-abbe-456a033cbb13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:24.045904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f08264-932a-426c-abbe-456a033cbb13 None admin] RBAC: Adding query filter params (role.id=dd4438dfc8ea468db528cc4cda58f3b8, user.id=1af75e1b60414d0ab0abe2ad14b95495, scope.system=all) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:24.046214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f08264-932a-426c-abbe-456a033cbb13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:24.047696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f08264-932a-426c-abbe-456a033cbb13 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:24.062585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 184/357] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jul 5 15:00:23 2022] GET /identity/v3/role_assignments?user.id=1af75e1b60414d0ab0abe2ad14b95495&scope.system=all&role.id=dd4438dfc8ea468db528cc4cda58f3b8 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:24.063404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66046f9-3142-4a77-a941-470d86156f02 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:24.063702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66046f9-3142-4a77-a941-470d86156f02 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:24.063986 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66046f9-3142-4a77-a941-470d86156f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:24.083270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-e66046f9-3142-4a77-a941-470d86156f02 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:24.084659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e66046f9-3142-4a77-a941-470d86156f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:24.102368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 174/358] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:24.103489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391018f6-453a-4ab3-a9ce-899762bd1240 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:24.171408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 185/359] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:24.172936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7092b-294c-472b-a6be-4236f8517a18 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:24.173240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7092b-294c-472b-a6be-4236f8517a18 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:24.173528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7092b-294c-472b-a6be-4236f8517a18 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:24.174967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 186/360] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:24.175671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7092b-294c-472b-a6be-4236f8517a18 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:24.216109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:24.254799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391018f6-453a-4ab3-a9ce-899762bd1240 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:24.279676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:24.295475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c80849-4526-4168-822b-d40ad334de1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:24.296249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c80849-4526-4168-822b-d40ad334de1d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:24.296542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c80849-4526-4168-822b-d40ad334de1d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:24.296825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c80849-4526-4168-822b-d40ad334de1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:24.307631 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7092b-294c-472b-a6be-4236f8517a18 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:24.353983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c638ae37-0fae-4a58-ac69-c065b69ebb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:24.354809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ae37-0fae-4a58-ac69-c065b69ebb6e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:24.355375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ae37-0fae-4a58-ac69-c065b69ebb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:24.355556 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c638ae37-0fae-4a58-ac69-c065b69ebb6e None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:24.356620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638ae37-0fae-4a58-ac69-c065b69ebb6e None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:24.358113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c638ae37-0fae-4a58-ac69-c065b69ebb6e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:24.360496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 187/361] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:24 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:24.361610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4319f44-5b32-48ae-8539-8a129ef2330c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:24.361718 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4319f44-5b32-48ae-8539-8a129ef2330c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:24.362191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4319f44-5b32-48ae-8539-8a129ef2330c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:24.380942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d4319f44-5b32-48ae-8539-8a129ef2330c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:24.382145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d4319f44-5b32-48ae-8539-8a129ef2330c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:24.390989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c80849-4526-4168-822b-d40ad334de1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:24.392973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c80849-4526-4168-822b-d40ad334de1d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:24.466279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 188/362] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:24.467680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4319f44-5b32-48ae-8539-8a129ef2330c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:24.487459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:24.525435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 175/363] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:24.526002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c80849-4526-4168-822b-d40ad334de1d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:24.551137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:24.595935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4319f44-5b32-48ae-8539-8a129ef2330c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:24.639478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53596079-55fc-4744-9cd3-6df7f3fcfb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:24.640347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53596079-55fc-4744-9cd3-6df7f3fcfb12 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:24.642718 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53596079-55fc-4744-9cd3-6df7f3fcfb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:24.643009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53596079-55fc-4744-9cd3-6df7f3fcfb12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:24.649283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c80849-4526-4168-822b-d40ad334de1d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:24.655951 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53596079-55fc-4744-9cd3-6df7f3fcfb12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-53596079-55fc-4744-9cd3-6df7f3fcfb12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:24.665731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:24.666620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:24.667476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:24.668110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 189/364] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:00:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:24.668650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53596079-55fc-4744-9cd3-6df7f3fcfb12 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:24.700186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dea2e-4d9c-4c68-8efb-d41f23b00105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:24.702201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dea2e-4d9c-4c68-8efb-d41f23b00105 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:24.702486 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dea2e-4d9c-4c68-8efb-d41f23b00105 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:24.702764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6dea2e-4d9c-4c68-8efb-d41f23b00105 None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:24.708127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6dea2e-4d9c-4c68-8efb-d41f23b00105 None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:24.708266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6dea2e-4d9c-4c68-8efb-d41f23b00105 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:24.710088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 176/365] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:24 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:24.711412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dea2e-4d9c-4c68-8efb-d41f23b00105 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:24.776573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:24.808766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53596079-55fc-4744-9cd3-6df7f3fcfb12 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:24.828970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:24.846875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6dea2e-4d9c-4c68-8efb-d41f23b00105 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:24.852599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e8aca-5011-4309-8a3f-50c41bfed079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:24.853481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8aca-5011-4309-8a3f-50c41bfed079 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:24.853832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8aca-5011-4309-8a3f-50c41bfed079 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:24.854177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e8aca-5011-4309-8a3f-50c41bfed079 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:24.866480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e8aca-5011-4309-8a3f-50c41bfed079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-9a9e8aca-5011-4309-8a3f-50c41bfed079 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:24.874806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:24.876046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 190/366] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:24.877151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e8aca-5011-4309-8a3f-50c41bfed079 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:24.901155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649945ce-7db7-431a-b2cc-91e128eb1165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:24.901860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649945ce-7db7-431a-b2cc-91e128eb1165 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:24.902695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649945ce-7db7-431a-b2cc-91e128eb1165 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:24.903003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649945ce-7db7-431a-b2cc-91e128eb1165 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:24.904288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649945ce-7db7-431a-b2cc-91e128eb1165 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:24.905749 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649945ce-7db7-431a-b2cc-91e128eb1165 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:24.908249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 177/367] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:24 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:24.911637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649945ce-7db7-431a-b2cc-91e128eb1165 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:25.008759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e8aca-5011-4309-8a3f-50c41bfed079 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:25.039117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649945ce-7db7-431a-b2cc-91e128eb1165 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:25.050890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335390ab-7176-45fc-aace-1741e859f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:25.052171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335390ab-7176-45fc-aace-1741e859f564 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:25.052457 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335390ab-7176-45fc-aace-1741e859f564 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:25.052738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335390ab-7176-45fc-aace-1741e859f564 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:25.053511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335390ab-7176-45fc-aace-1741e859f564 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:25.053823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335390ab-7176-45fc-aace-1741e859f564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:25.055566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335390ab-7176-45fc-aace-1741e859f564 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:25.061245 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:25.067589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 191/368] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:25.068560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335390ab-7176-45fc-aace-1741e859f564 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:25.086638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec18146-583a-4554-af7c-bbb27ac3d331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:25.087506 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec18146-583a-4554-af7c-bbb27ac3d331 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:25.087796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec18146-583a-4554-af7c-bbb27ac3d331 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:25.088078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec18146-583a-4554-af7c-bbb27ac3d331 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:25.088930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec18146-583a-4554-af7c-bbb27ac3d331 None admin] RBAC: Adding query filter params (group.id=670d33c6c7f24ab1b3c3e3880dae2beb, role.id=6e32448ba01f41bcafb4cb0a839691fe, scope.project.id=96f8d6ac74154c299166a6e2bf295a72) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:25.089250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec18146-583a-4554-af7c-bbb27ac3d331 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:25.090698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec18146-583a-4554-af7c-bbb27ac3d331 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:25.103305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 178/369] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 5 15:00:24 2022] GET /identity/v3/role_assignments?group.id=670d33c6c7f24ab1b3c3e3880dae2beb&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:25.104867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec18146-583a-4554-af7c-bbb27ac3d331 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:25.139701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:25.201118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335390ab-7176-45fc-aace-1741e859f564 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:25.244978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42305c-8442-4bf6-b27c-987122a3e8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:25.245953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42305c-8442-4bf6-b27c-987122a3e8fc None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:25.246335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42305c-8442-4bf6-b27c-987122a3e8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:25.246710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42305c-8442-4bf6-b27c-987122a3e8fc None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:25.247647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42305c-8442-4bf6-b27c-987122a3e8fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:25.248059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42305c-8442-4bf6-b27c-987122a3e8fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:25.249687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42305c-8442-4bf6-b27c-987122a3e8fc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:25.260126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 192/370] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:25.261571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42305c-8442-4bf6-b27c-987122a3e8fc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:25.262927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec18146-583a-4554-af7c-bbb27ac3d331 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:25.306987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb84ae-f172-42d8-864d-bce137f958b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:25.308160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb84ae-f172-42d8-864d-bce137f958b6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:25.308596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb84ae-f172-42d8-864d-bce137f958b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:25.309023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb84ae-f172-42d8-864d-bce137f958b6 None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:25.311414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb84ae-f172-42d8-864d-bce137f958b6 None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:25.313444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb84ae-f172-42d8-864d-bce137f958b6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:25.316236 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 179/371] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:25 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:25.317353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb84ae-f172-42d8-864d-bce137f958b6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:25.340322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:25.384299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42305c-8442-4bf6-b27c-987122a3e8fc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:25.423420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589f5d5-85e5-486a-8b44-79d1461ca298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:25.424259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589f5d5-85e5-486a-8b44-79d1461ca298 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:25.424792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589f5d5-85e5-486a-8b44-79d1461ca298 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:25.425005 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589f5d5-85e5-486a-8b44-79d1461ca298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:25.432245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:25.452561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb84ae-f172-42d8-864d-bce137f958b6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:25.506025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3ec17-8462-45a4-b646-3d7f639123d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:25.507244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3ec17-8462-45a4-b646-3d7f639123d5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:25.508073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3ec17-8462-45a4-b646-3d7f639123d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:25.508755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3ec17-8462-45a4-b646-3d7f639123d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:25.515903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589f5d5-85e5-486a-8b44-79d1461ca298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:25.517538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a589f5d5-85e5-486a-8b44-79d1461ca298 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:25.591372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:25.602962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3ec17-8462-45a4-b646-3d7f639123d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:25.604736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3ec17-8462-45a4-b646-3d7f639123d5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:25.652138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 193/372] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:25.653879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589f5d5-85e5-486a-8b44-79d1461ca298 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:25.729201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:25.750363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 180/373] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:25.751293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3ec17-8462-45a4-b646-3d7f639123d5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:25.774482 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a589f5d5-85e5-486a-8b44-79d1461ca298 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:25.815256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b444945-0d83-4450-8c00-e04cc950e86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:25.815907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b444945-0d83-4450-8c00-e04cc950e86c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:25.816200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b444945-0d83-4450-8c00-e04cc950e86c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:25.816477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b444945-0d83-4450-8c00-e04cc950e86c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:25.817614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b444945-0d83-4450-8c00-e04cc950e86c None admin] RBAC: Adding query filter params (group.id=670d33c6c7f24ab1b3c3e3880dae2beb, role.id=4301bced0259446cafb79118a53a015e, scope.project.id=a2c6e66812894558944eb8f6f156896d) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:25.817930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b444945-0d83-4450-8c00-e04cc950e86c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:25.819712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b444945-0d83-4450-8c00-e04cc950e86c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:25.825889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 194/374] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 5 15:00:25 2022] GET /identity/v3/role_assignments?group.id=670d33c6c7f24ab1b3c3e3880dae2beb&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=4301bced0259446cafb79118a53a015e => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:25.826945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b444945-0d83-4450-8c00-e04cc950e86c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:25.893287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:25.957789 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3ec17-8462-45a4-b646-3d7f639123d5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:26.021357 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdefb6b-7173-4a98-991d-ca60286109ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:26.022822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdefb6b-7173-4a98-991d-ca60286109ca None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:26.023143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdefb6b-7173-4a98-991d-ca60286109ca None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:26.023432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdefb6b-7173-4a98-991d-ca60286109ca None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:26.024767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdefb6b-7173-4a98-991d-ca60286109ca None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:26.026812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdefb6b-7173-4a98-991d-ca60286109ca None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:26.029264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 181/375] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:25 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:26.030724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b444945-0d83-4450-8c00-e04cc950e86c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:26.031468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8265222-7a8b-4db0-91e5-49a29efa0ed8 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:26.031762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8265222-7a8b-4db0-91e5-49a29efa0ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:26.032047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8265222-7a8b-4db0-91e5-49a29efa0ed8 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:26.036545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 182/376] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:26.037328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8265222-7a8b-4db0-91e5-49a29efa0ed8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:26.082161 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fef3ea-7ddb-46d1-a827-f62e2c2dd44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:26.083406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fef3ea-7ddb-46d1-a827-f62e2c2dd44a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:26.083902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fef3ea-7ddb-46d1-a827-f62e2c2dd44a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:26.084411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fef3ea-7ddb-46d1-a827-f62e2c2dd44a None admin] PATH_INFO: `/v3/projects/77588a8e554a4d32b7cfdcc698266c19` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:26.085814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fef3ea-7ddb-46d1-a827-f62e2c2dd44a None admin] RBAC: Authorizing `identity:get_project(project_id=77588a8e554a4d32b7cfdcc698266c19)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:26.087514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fef3ea-7ddb-46d1-a827-f62e2c2dd44a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:26.089756 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 195/377] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:25 2022] GET /identity/v3/projects/77588a8e554a4d32b7cfdcc698266c19 => generated 344 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:26.091744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fdd77f-031e-4660-aee1-747f438899f3 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:26.092290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fdd77f-031e-4660-aee1-747f438899f3 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:26.092834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fdd77f-031e-4660-aee1-747f438899f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:26.110404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b0fdd77f-031e-4660-aee1-747f438899f3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:26.111961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b0fdd77f-031e-4660-aee1-747f438899f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:26.112870 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:26.161866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8265222-7a8b-4db0-91e5-49a29efa0ed8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:26.200952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12b70a-8448-4541-930b-04704d6a498a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:26.201942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 196/378] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:26.203105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fdd77f-031e-4660-aee1-747f438899f3 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:26.203381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12b70a-8448-4541-930b-04704d6a498a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:26.203687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12b70a-8448-4541-930b-04704d6a498a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:26.203976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12b70a-8448-4541-930b-04704d6a498a None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:26.205205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12b70a-8448-4541-930b-04704d6a498a None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:26.206655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12b70a-8448-4541-930b-04704d6a498a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:26.217195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 183/379] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:26 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:26.225805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12b70a-8448-4541-930b-04704d6a498a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:26.236691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:26.319568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fdd77f-031e-4660-aee1-747f438899f3 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:26.333633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12b70a-8448-4541-930b-04704d6a498a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:26.360363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b786d-fb36-40e6-be10-ee304258c3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:26.361077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:26.362981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b786d-fb36-40e6-be10-ee304258c3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:26.363498 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b786d-fb36-40e6-be10-ee304258c3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:26.363943 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b786d-fb36-40e6-be10-ee304258c3f2 None admin] PATH_INFO: `/v3/groups/c94e9ed8cc4f4b968d399dd168f1db29` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:26.372368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b786d-fb36-40e6-be10-ee304258c3f2 None admin] RBAC: Authorizing `identity:get_group(group_id=c94e9ed8cc4f4b968d399dd168f1db29)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:26.374049 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e821b-c9d9-41bf-967c-e76802806c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:26.374549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b786d-fb36-40e6-be10-ee304258c3f2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:26.374729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e821b-c9d9-41bf-967c-e76802806c6b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:26.375005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e821b-c9d9-41bf-967c-e76802806c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:26.375331 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e821b-c9d9-41bf-967c-e76802806c6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:26.376200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e821b-c9d9-41bf-967c-e76802806c6b None admin] RBAC: Adding query filter params (group.id=670d33c6c7f24ab1b3c3e3880dae2beb, role.id=4301bced0259446cafb79118a53a015e, scope.project.id=96f8d6ac74154c299166a6e2bf295a72) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:26.376517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e821b-c9d9-41bf-967c-e76802806c6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:26.378013 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e821b-c9d9-41bf-967c-e76802806c6b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:26.378132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 197/380] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:26 2022] GET /identity/v3/groups/c94e9ed8cc4f4b968d399dd168f1db29 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:26.379183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b786d-fb36-40e6-be10-ee304258c3f2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:26.384676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 184/381] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 5 15:00:26 2022] GET /identity/v3/role_assignments?group.id=670d33c6c7f24ab1b3c3e3880dae2beb&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=4301bced0259446cafb79118a53a015e => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:26.385496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e821b-c9d9-41bf-967c-e76802806c6b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:26.572204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:26.600950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b786d-fb36-40e6-be10-ee304258c3f2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:26.619306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e821b-c9d9-41bf-967c-e76802806c6b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:26.669510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f03d2f-2e04-452f-bfc8-b39a8f9f24da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:26.670673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03d2f-2e04-452f-bfc8-b39a8f9f24da None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:26.671194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03d2f-2e04-452f-bfc8-b39a8f9f24da None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:26.671697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f03d2f-2e04-452f-bfc8-b39a8f9f24da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:26.687083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcf628-0d84-4f96-9129-420203b6ab86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:26.688086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcf628-0d84-4f96-9129-420203b6ab86 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:26.691003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcf628-0d84-4f96-9129-420203b6ab86 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:26.691537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcf628-0d84-4f96-9129-420203b6ab86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:26.692599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcf628-0d84-4f96-9129-420203b6ab86 None admin] RBAC: Adding query filter params (group.id=c94e9ed8cc4f4b968d399dd168f1db29, role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=77588a8e554a4d32b7cfdcc698266c19) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:26.693468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcf628-0d84-4f96-9129-420203b6ab86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:26.694184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f03d2f-2e04-452f-bfc8-b39a8f9f24da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:26.695870 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcf628-0d84-4f96-9129-420203b6ab86 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-95f03d2f-2e04-452f-bfc8-b39a8f9f24da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:26.704289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:26.705187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:26.705977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:26.713433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 185/382] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 5 15:00:26 2022] GET /identity/v3/role_assignments?group.id=c94e9ed8cc4f4b968d399dd168f1db29&scope.project.id=77588a8e554a4d32b7cfdcc698266c19&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 260 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:26.714315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 198/383] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:26.728600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcf628-0d84-4f96-9129-420203b6ab86 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:26.807319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:26.885571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcf628-0d84-4f96-9129-420203b6ab86 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:26.931991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2416d8-db36-496f-9398-73bc225fc221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:26.933090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2416d8-db36-496f-9398-73bc225fc221 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:26.933623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2416d8-db36-496f-9398-73bc225fc221 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:26.934124 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2416d8-db36-496f-9398-73bc225fc221 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:26.935610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2416d8-db36-496f-9398-73bc225fc221 None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:26.936121 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2416d8-db36-496f-9398-73bc225fc221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:26.937769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2416d8-db36-496f-9398-73bc225fc221 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:26.952307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 186/384] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:26.957630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f03d2f-2e04-452f-bfc8-b39a8f9f24da None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:27.071941 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f03d2f-2e04-452f-bfc8-b39a8f9f24da None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:27.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0873a-0a61-485e-b37c-de9e231eb027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:27.112176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0873a-0a61-485e-b37c-de9e231eb027 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:27.112463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0873a-0a61-485e-b37c-de9e231eb027 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:27.112739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0873a-0a61-485e-b37c-de9e231eb027 None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:27.113906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0873a-0a61-485e-b37c-de9e231eb027 None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:27.115379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0873a-0a61-485e-b37c-de9e231eb027 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:27.117010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 199/385] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:26 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:27.122803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2416d8-db36-496f-9398-73bc225fc221 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:27.275855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2416d8-db36-496f-9398-73bc225fc221 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:27.298609 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:27.326073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebe6be-4ada-4e7d-a156-d83ba003523a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:27.327226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe6be-4ada-4e7d-a156-d83ba003523a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:27.327515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe6be-4ada-4e7d-a156-d83ba003523a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:27.327791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebe6be-4ada-4e7d-a156-d83ba003523a None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:27.331795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebe6be-4ada-4e7d-a156-d83ba003523a None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:27.333248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebe6be-4ada-4e7d-a156-d83ba003523a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:27.335441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 187/386] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:27 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:27.341146 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0873a-0a61-485e-b37c-de9e231eb027 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:27.368890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:27.446625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d596db4-d6d7-4b5f-a50d-a48a7bc7742c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:27.446949 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d596db4-d6d7-4b5f-a50d-a48a7bc7742c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:27.447150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d596db4-d6d7-4b5f-a50d-a48a7bc7742c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:27.448780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 188/387] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:27.454702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005513a-c30d-4ee8-b1e7-40d340508a94 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:27.454995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005513a-c30d-4ee8-b1e7-40d340508a94 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:27.455310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005513a-c30d-4ee8-b1e7-40d340508a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:27.474760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1005513a-c30d-4ee8-b1e7-40d340508a94 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:27.476090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0873a-0a61-485e-b37c-de9e231eb027 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:27.480398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1005513a-c30d-4ee8-b1e7-40d340508a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:27.536426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661af97-c979-42ed-af27-d406c54c40bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:27.537318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661af97-c979-42ed-af27-d406c54c40bd None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:27.537604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661af97-c979-42ed-af27-d406c54c40bd None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:27.537901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661af97-c979-42ed-af27-d406c54c40bd None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d/groups/670d33c6c7f24ab1b3c3e3880dae2beb/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:27.544006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661af97-c979-42ed-af27-d406c54c40bd None admin] RBAC: Authorizing `identity:create_grant(project_id=a2c6e66812894558944eb8f6f156896d, group_id=670d33c6c7f24ab1b3c3e3880dae2beb, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:27.545536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661af97-c979-42ed-af27-d406c54c40bd None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:27.569509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 200/388] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jul 5 15:00:27 2022] PUT /identity/v3/projects/a2c6e66812894558944eb8f6f156896d/groups/670d33c6c7f24ab1b3c3e3880dae2beb/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:27.593778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 189/389] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:27.625429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661af97-c979-42ed-af27-d406c54c40bd None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:27.730728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:27.757592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661af97-c979-42ed-af27-d406c54c40bd None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:27.802647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091977e-486b-4deb-b61a-acd6019a329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:27.803880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091977e-486b-4deb-b61a-acd6019a329c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:27.804569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091977e-486b-4deb-b61a-acd6019a329c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:27.805343 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091977e-486b-4deb-b61a-acd6019a329c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:27.819608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091977e-486b-4deb-b61a-acd6019a329c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-e091977e-486b-4deb-b61a-acd6019a329c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:27.836099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:27.837643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:27.839035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:27.843767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 201/390] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:27.849723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005513a-c30d-4ee8-b1e7-40d340508a94 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:27.977481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005513a-c30d-4ee8-b1e7-40d340508a94 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:27.991429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:28.018613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a5d81-9d8c-40d7-b29f-76711848d7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:28.019814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a5d81-9d8c-40d7-b29f-76711848d7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:28.020322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a5d81-9d8c-40d7-b29f-76711848d7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:28.020826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a5d81-9d8c-40d7-b29f-76711848d7c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:28.022296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a5d81-9d8c-40d7-b29f-76711848d7c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:28.022860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a5d81-9d8c-40d7-b29f-76711848d7c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:28.024789 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a5d81-9d8c-40d7-b29f-76711848d7c9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:28.037737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 190/391] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:28.042933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091977e-486b-4deb-b61a-acd6019a329c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:28.160594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091977e-486b-4deb-b61a-acd6019a329c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:28.210337 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b8216-fdb1-4a1b-b46f-a5a27987e824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:28.211284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b8216-fdb1-4a1b-b46f-a5a27987e824 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:28.211617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b8216-fdb1-4a1b-b46f-a5a27987e824 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:28.211903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b8216-fdb1-4a1b-b46f-a5a27987e824 None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:28.213080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b8216-fdb1-4a1b-b46f-a5a27987e824 None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:28.214619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b8216-fdb1-4a1b-b46f-a5a27987e824 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:28.217140 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 202/392] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:28 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:28.222411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a5d81-9d8c-40d7-b29f-76711848d7c9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:28.353966 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a5d81-9d8c-40d7-b29f-76711848d7c9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:28.389239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc77f6-1392-4767-aa2e-189d2a310585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:28.390110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc77f6-1392-4767-aa2e-189d2a310585 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:28.390392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc77f6-1392-4767-aa2e-189d2a310585 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:28.390667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc77f6-1392-4767-aa2e-189d2a310585 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:28.391879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc77f6-1392-4767-aa2e-189d2a310585 None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:28.393324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc77f6-1392-4767-aa2e-189d2a310585 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:28.395021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 191/393] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:28 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:28.400690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b8216-fdb1-4a1b-b46f-a5a27987e824 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:28.414044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d54bd-8bf5-463a-a6cc-12e745afe1da None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:28.414332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d54bd-8bf5-463a-a6cc-12e745afe1da None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:28.414613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d54bd-8bf5-463a-a6cc-12e745afe1da None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:28.418752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 192/394] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:28.422580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7094cf-9216-418f-b1aa-c8812afb87ee None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:28.422866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7094cf-9216-418f-b1aa-c8812afb87ee None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:28.423190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7094cf-9216-418f-b1aa-c8812afb87ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:28.423410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:28.451362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7c7094cf-9216-418f-b1aa-c8812afb87ee None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:28.452803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7c7094cf-9216-418f-b1aa-c8812afb87ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:28.527123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:28.589529 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 193/395] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:28.591329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b8216-fdb1-4a1b-b46f-a5a27987e824 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:28.624624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7094cf-9216-418f-b1aa-c8812afb87ee None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:28.641861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26423da3-9f5a-4608-a705-57b91c89b3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:28.643400 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26423da3-9f5a-4608-a705-57b91c89b3dc None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:28.643680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26423da3-9f5a-4608-a705-57b91c89b3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:28.643955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26423da3-9f5a-4608-a705-57b91c89b3dc None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72/groups/670d33c6c7f24ab1b3c3e3880dae2beb/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:28.648142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26423da3-9f5a-4608-a705-57b91c89b3dc None admin] RBAC: Authorizing `identity:create_grant(project_id=96f8d6ac74154c299166a6e2bf295a72, group_id=670d33c6c7f24ab1b3c3e3880dae2beb, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:28.651969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26423da3-9f5a-4608-a705-57b91c89b3dc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:28.675540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 203/396] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jul 5 15:00:28 2022] PUT /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72/groups/670d33c6c7f24ab1b3c3e3880dae2beb/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:28.767265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7094cf-9216-418f-b1aa-c8812afb87ee None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:28.813799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc156d-b3cf-436f-b521-2bfe3182dd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:28.814871 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc156d-b3cf-436f-b521-2bfe3182dd03 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:28.815341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc156d-b3cf-436f-b521-2bfe3182dd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:28.815844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc156d-b3cf-436f-b521-2bfe3182dd03 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:28.838436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc156d-b3cf-436f-b521-2bfe3182dd03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-65bc156d-b3cf-436f-b521-2bfe3182dd03 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:28.851726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:28.852876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:28.853843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:28.857076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 194/397] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:00:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:28.862665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26423da3-9f5a-4608-a705-57b91c89b3dc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:28.948767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:28.998514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26423da3-9f5a-4608-a705-57b91c89b3dc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:29.046679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954e0ab-b11e-4a6a-9eb3-f7eecfdf6daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:29.047669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e0ab-b11e-4a6a-9eb3-f7eecfdf6daf None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:29.047957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e0ab-b11e-4a6a-9eb3-f7eecfdf6daf None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:29.048232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e0ab-b11e-4a6a-9eb3-f7eecfdf6daf None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:29.049063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954e0ab-b11e-4a6a-9eb3-f7eecfdf6daf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:29.049300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954e0ab-b11e-4a6a-9eb3-f7eecfdf6daf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:29.050696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954e0ab-b11e-4a6a-9eb3-f7eecfdf6daf None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:29.059441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 204/398] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:29.065468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc156d-b3cf-436f-b521-2bfe3182dd03 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:29.134262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:29.238350 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc156d-b3cf-436f-b521-2bfe3182dd03 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:29.287086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e338b8-0912-46a4-a139-1cc8cad2aec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:29.287988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e338b8-0912-46a4-a139-1cc8cad2aec0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:29.288110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6af34b-bc62-4ae3-bed9-a7c47375772c None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:29.288274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e338b8-0912-46a4-a139-1cc8cad2aec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:29.288373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6af34b-bc62-4ae3-bed9-a7c47375772c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:29.288559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e338b8-0912-46a4-a139-1cc8cad2aec0 None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:29.288655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6af34b-bc62-4ae3-bed9-a7c47375772c None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:29.289802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e338b8-0912-46a4-a139-1cc8cad2aec0 None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:29.290372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 205/399] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:29.292434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e338b8-0912-46a4-a139-1cc8cad2aec0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:29.294592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 195/400] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:29 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:29.300065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef9f16-2067-4bb1-b122-c101c2b201cb None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:29.300358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef9f16-2067-4bb1-b122-c101c2b201cb None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:29.300641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef9f16-2067-4bb1-b122-c101c2b201cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:29.303571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e338b8-0912-46a4-a139-1cc8cad2aec0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:29.326120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c1ef9f16-2067-4bb1-b122-c101c2b201cb None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:29.331757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c1ef9f16-2067-4bb1-b122-c101c2b201cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:29.435853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e338b8-0912-46a4-a139-1cc8cad2aec0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:29.443369 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:29.472945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aabb20-a7ee-4f98-9d60-5c57b71be25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:29.473926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aabb20-a7ee-4f98-9d60-5c57b71be25c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:29.474217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aabb20-a7ee-4f98-9d60-5c57b71be25c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:29.474499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aabb20-a7ee-4f98-9d60-5c57b71be25c None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:29.475728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aabb20-a7ee-4f98-9d60-5c57b71be25c None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:29.477172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aabb20-a7ee-4f98-9d60-5c57b71be25c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:29.479302 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 196/401] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:29 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:29.486153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aabb20-a7ee-4f98-9d60-5c57b71be25c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:29.511239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 206/402] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:29.539961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef9f16-2067-4bb1-b122-c101c2b201cb None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:29.647141 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:29.686261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aabb20-a7ee-4f98-9d60-5c57b71be25c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:29.710989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef9f16-2067-4bb1-b122-c101c2b201cb None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:29.722000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9632ad-25f6-4775-b395-e61bc75a7945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:29.722889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9632ad-25f6-4775-b395-e61bc75a7945 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:29.723222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9632ad-25f6-4775-b395-e61bc75a7945 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:29.723504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9632ad-25f6-4775-b395-e61bc75a7945 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d/groups/670d33c6c7f24ab1b3c3e3880dae2beb/roles/4301bced0259446cafb79118a53a015e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:29.725755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9632ad-25f6-4775-b395-e61bc75a7945 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2c6e66812894558944eb8f6f156896d, group_id=670d33c6c7f24ab1b3c3e3880dae2beb, role_id=4301bced0259446cafb79118a53a015e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:29.727233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9632ad-25f6-4775-b395-e61bc75a7945 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:29.744078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 197/403] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jul 5 15:00:29 2022] PUT /identity/v3/projects/a2c6e66812894558944eb8f6f156896d/groups/670d33c6c7f24ab1b3c3e3880dae2beb/roles/4301bced0259446cafb79118a53a015e => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:00:29.745700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225f4c-c641-42cc-866d-081497f35750 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:29.746171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225f4c-c641-42cc-866d-081497f35750 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:29.746643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c225f4c-c641-42cc-866d-081497f35750 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:29.753001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 198/404] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:29.754812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e7312-fe1c-4bdc-a4be-544134756452 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:29.755345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e7312-fe1c-4bdc-a4be-544134756452 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:29.755839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e7312-fe1c-4bdc-a4be-544134756452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:29.770434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26d1d9-477b-4d3c-94bb-b73dff2be496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:29.771333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d1d9-477b-4d3c-94bb-b73dff2be496 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:29.771692 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d1d9-477b-4d3c-94bb-b73dff2be496 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:29.771997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26d1d9-477b-4d3c-94bb-b73dff2be496 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:29.776368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3c1e7312-fe1c-4bdc-a4be-544134756452 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:29.777862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3c1e7312-fe1c-4bdc-a4be-544134756452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:29.804471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26d1d9-477b-4d3c-94bb-b73dff2be496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-8d26d1d9-477b-4d3c-94bb-b73dff2be496 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:29.818830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:29.819897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:29.820694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:29.824019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 207/405] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:00:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:29.824598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:29.829914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26d1d9-477b-4d3c-94bb-b73dff2be496 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:29.916885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 199/406] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:29.945071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e7312-fe1c-4bdc-a4be-544134756452 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:30.011965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26d1d9-477b-4d3c-94bb-b73dff2be496 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:30.035933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:30.074574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa707f7-44fa-4148-ae03-a79b9af0ccd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:30.075823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa707f7-44fa-4148-ae03-a79b9af0ccd5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:30.076117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa707f7-44fa-4148-ae03-a79b9af0ccd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:30.076641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa707f7-44fa-4148-ae03-a79b9af0ccd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:30.077441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa707f7-44fa-4148-ae03-a79b9af0ccd5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:30.077754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa707f7-44fa-4148-ae03-a79b9af0ccd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:30.079246 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa707f7-44fa-4148-ae03-a79b9af0ccd5 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:30.096840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 208/407] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:30.108113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa707f7-44fa-4148-ae03-a79b9af0ccd5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:30.110953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e7312-fe1c-4bdc-a4be-544134756452 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:30.155711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896fb26-e121-4edc-a38b-b6cd5c72439a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:30.156512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896fb26-e121-4edc-a38b-b6cd5c72439a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:30.156796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896fb26-e121-4edc-a38b-b6cd5c72439a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:30.157075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896fb26-e121-4edc-a38b-b6cd5c72439a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:30.180776 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9896fb26-e121-4edc-a38b-b6cd5c72439a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-9896fb26-e121-4edc-a38b-b6cd5c72439a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:30.189040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:30.189920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:30.190704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:30.192578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 200/408] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:00:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:30.204649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896fb26-e121-4edc-a38b-b6cd5c72439a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:30.239891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:30.283025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa707f7-44fa-4148-ae03-a79b9af0ccd5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:30.328163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec5326-c725-4a1e-98e0-59ccc232bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:30.328958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec5326-c725-4a1e-98e0-59ccc232bd52 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:30.329410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec5326-c725-4a1e-98e0-59ccc232bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:30.329697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec5326-c725-4a1e-98e0-59ccc232bd52 None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:30.331555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec5326-c725-4a1e-98e0-59ccc232bd52 None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:30.333249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec5326-c725-4a1e-98e0-59ccc232bd52 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:30.335599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 209/409] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:30 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:30.336862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7e2e9-24b2-410a-ac74-c7c547a9dc83 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:30.337156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7e2e9-24b2-410a-ac74-c7c547a9dc83 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:30.337437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7e2e9-24b2-410a-ac74-c7c547a9dc83 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:30.343124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 210/410] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:30.343946 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7e2e9-24b2-410a-ac74-c7c547a9dc83 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:30.358599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9896fb26-e121-4edc-a38b-b6cd5c72439a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:30.401599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf2cc4-763a-4b08-bd79-5c1a5f8ae661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:30.402281 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf2cc4-763a-4b08-bd79-5c1a5f8ae661 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:30.402566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf2cc4-763a-4b08-bd79-5c1a5f8ae661 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:30.402847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf2cc4-763a-4b08-bd79-5c1a5f8ae661 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:30.404712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf2cc4-763a-4b08-bd79-5c1a5f8ae661 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:30.405028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf2cc4-763a-4b08-bd79-5c1a5f8ae661 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:30.406691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf2cc4-763a-4b08-bd79-5c1a5f8ae661 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:30.409024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:30.420191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 201/411] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:30.422287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ef925-3ba4-4555-8b79-9e023c471f6b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:30.422582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ef925-3ba4-4555-8b79-9e023c471f6b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:30.422864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ef925-3ba4-4555-8b79-9e023c471f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:30.443298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e77ef925-3ba4-4555-8b79-9e023c471f6b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:30.444490 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e77ef925-3ba4-4555-8b79-9e023c471f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:30.477398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7e2e9-24b2-410a-ac74-c7c547a9dc83 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:30.522232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed02e4-0ddf-44a7-aa79-6fa91f7e2b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:30.523784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed02e4-0ddf-44a7-aa79-6fa91f7e2b70 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:30.524416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed02e4-0ddf-44a7-aa79-6fa91f7e2b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:30.525220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed02e4-0ddf-44a7-aa79-6fa91f7e2b70 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:30.526925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed02e4-0ddf-44a7-aa79-6fa91f7e2b70 None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:30.529380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed02e4-0ddf-44a7-aa79-6fa91f7e2b70 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:30.535074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 202/412] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:30.535074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ef925-3ba4-4555-8b79-9e023c471f6b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:30.542543 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 211/413] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:30 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:30.542836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed02e4-0ddf-44a7-aa79-6fa91f7e2b70 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:30.560821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:30.665387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ef925-3ba4-4555-8b79-9e023c471f6b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:30.683229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed02e4-0ddf-44a7-aa79-6fa91f7e2b70 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:30.697801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:30.714450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218a709-89dd-4a56-aee5-f24a3ba810e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:30.716692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218a709-89dd-4a56-aee5-f24a3ba810e5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:30.717411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218a709-89dd-4a56-aee5-f24a3ba810e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:30.717948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218a709-89dd-4a56-aee5-f24a3ba810e5 None admin] PATH_INFO: `/v3/groups/c94e9ed8cc4f4b968d399dd168f1db29` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:30.719447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218a709-89dd-4a56-aee5-f24a3ba810e5 None admin] RBAC: Authorizing `identity:get_group(group_id=c94e9ed8cc4f4b968d399dd168f1db29)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:30.721279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218a709-89dd-4a56-aee5-f24a3ba810e5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:30.721519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1659a46-744d-49c0-aefc-ec6262840235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:30.722506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1659a46-744d-49c0-aefc-ec6262840235 None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:30.722887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1659a46-744d-49c0-aefc-ec6262840235 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:30.723291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1659a46-744d-49c0-aefc-ec6262840235 None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72/groups/670d33c6c7f24ab1b3c3e3880dae2beb/roles/4301bced0259446cafb79118a53a015e` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:30.724050 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 203/414] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:30 2022] GET /identity/v3/groups/c94e9ed8cc4f4b968d399dd168f1db29 => generated 229 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:30.725034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218a709-89dd-4a56-aee5-f24a3ba810e5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:30.726064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1659a46-744d-49c0-aefc-ec6262840235 None admin] RBAC: Authorizing `identity:create_grant(project_id=96f8d6ac74154c299166a6e2bf295a72, group_id=670d33c6c7f24ab1b3c3e3880dae2beb, role_id=4301bced0259446cafb79118a53a015e)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:30.727668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1659a46-744d-49c0-aefc-ec6262840235 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:30.748508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 212/415] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jul 5 15:00:30 2022] PUT /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72/groups/670d33c6c7f24ab1b3c3e3880dae2beb/roles/4301bced0259446cafb79118a53a015e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:30.749731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1659a46-744d-49c0-aefc-ec6262840235 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:30.879353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1659a46-744d-49c0-aefc-ec6262840235 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:30.926849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:30.928659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646511a-bb60-432a-a4d6-95866b5f8960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:30.929801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646511a-bb60-432a-a4d6-95866b5f8960 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:30.930258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646511a-bb60-432a-a4d6-95866b5f8960 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:30.930689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646511a-bb60-432a-a4d6-95866b5f8960 None admin] PATH_INFO: `/v3/projects/77588a8e554a4d32b7cfdcc698266c19` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:30.933874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646511a-bb60-432a-a4d6-95866b5f8960 None admin] RBAC: Authorizing `identity:get_project(project_id=77588a8e554a4d32b7cfdcc698266c19)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:30.936395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646511a-bb60-432a-a4d6-95866b5f8960 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:30.941690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 213/416] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:30 2022] GET /identity/v3/projects/77588a8e554a4d32b7cfdcc698266c19 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:30.948030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646511a-bb60-432a-a4d6-95866b5f8960 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:30.971319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218a709-89dd-4a56-aee5-f24a3ba810e5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:31.009801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:31.017983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2a9b6-ac99-494f-b1dd-bff5e53d83b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:31.019613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2a9b6-ac99-494f-b1dd-bff5e53d83b6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:31.020076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2a9b6-ac99-494f-b1dd-bff5e53d83b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:31.020501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2a9b6-ac99-494f-b1dd-bff5e53d83b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:31.036205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2a9b6-ac99-494f-b1dd-bff5e53d83b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-bbe2a9b6-ac99-494f-b1dd-bff5e53d83b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:31.044566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:31.045471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:31.046288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 204/417] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:31.051874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2a9b6-ac99-494f-b1dd-bff5e53d83b6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:31.085314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646511a-bb60-432a-a4d6-95866b5f8960 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:31.125718 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1025a27-cd81-4f2b-bbbe-473ffc22da75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:31.126717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1025a27-cd81-4f2b-bbbe-473ffc22da75 None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:31.126999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1025a27-cd81-4f2b-bbbe-473ffc22da75 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:31.127302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1025a27-cd81-4f2b-bbbe-473ffc22da75 None admin] PATH_INFO: `/v3/projects/77588a8e554a4d32b7cfdcc698266c19/groups/c94e9ed8cc4f4b968d399dd168f1db29/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:31.129298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1025a27-cd81-4f2b-bbbe-473ffc22da75 None admin] RBAC: Authorizing `identity:create_grant(project_id=77588a8e554a4d32b7cfdcc698266c19, group_id=c94e9ed8cc4f4b968d399dd168f1db29, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:31.130941 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1025a27-cd81-4f2b-bbbe-473ffc22da75 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:31.147610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 214/418] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jul 5 15:00:30 2022] PUT /identity/v3/projects/77588a8e554a4d32b7cfdcc698266c19/groups/c94e9ed8cc4f4b968d399dd168f1db29/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:31.149505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f1092-1eb0-4381-8430-d5f1db0d677b None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:31.150021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f1092-1eb0-4381-8430-d5f1db0d677b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:31.150511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f1092-1eb0-4381-8430-d5f1db0d677b None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:31.152420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 215/419] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:31.165054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55d679-5156-48b8-942d-93d548c18264 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:31.165609 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55d679-5156-48b8-942d-93d548c18264 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:31.166051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55d679-5156-48b8-942d-93d548c18264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:31.202733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-fe55d679-5156-48b8-942d-93d548c18264 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:31.206335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-fe55d679-5156-48b8-942d-93d548c18264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:31.229743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2a9b6-ac99-494f-b1dd-bff5e53d83b6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:31.303529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 216/420] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:31.312447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d51508-2782-4ecd-9f6d-3944b747e2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:31.313685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d51508-2782-4ecd-9f6d-3944b747e2ed None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:31.314113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d51508-2782-4ecd-9f6d-3944b747e2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:31.314558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d51508-2782-4ecd-9f6d-3944b747e2ed None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:31.315500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d51508-2782-4ecd-9f6d-3944b747e2ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:31.315972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d51508-2782-4ecd-9f6d-3944b747e2ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:31.317503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d51508-2782-4ecd-9f6d-3944b747e2ed None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:31.328070 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:31.339829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 205/421] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:31.345811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55d679-5156-48b8-942d-93d548c18264 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:31.349483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d51508-2782-4ecd-9f6d-3944b747e2ed None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:31.397536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:31.469941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d51508-2782-4ecd-9f6d-3944b747e2ed None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:31.490044 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55d679-5156-48b8-942d-93d548c18264 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:31.517971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cc7b5-e751-4107-8fb6-5fe968e94f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:31.519238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cc7b5-e751-4107-8fb6-5fe968e94f61 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:31.519534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cc7b5-e751-4107-8fb6-5fe968e94f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:31.519814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cc7b5-e751-4107-8fb6-5fe968e94f61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:31.532198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca5969-1e37-4d0d-be7d-14adc7102884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:31.533196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca5969-1e37-4d0d-be7d-14adc7102884 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:31.533301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cc7b5-e751-4107-8fb6-5fe968e94f61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:31.533770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca5969-1e37-4d0d-be7d-14adc7102884 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:31.534238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca5969-1e37-4d0d-be7d-14adc7102884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f86cc7b5-e751-4107-8fb6-5fe968e94f61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:31.542229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:31.543192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:31.544637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 206/422] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:31.552045 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cc7b5-e751-4107-8fb6-5fe968e94f61 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:31.655476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:31.670143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca5969-1e37-4d0d-be7d-14adc7102884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:31.671989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca5969-1e37-4d0d-be7d-14adc7102884 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:31.708232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cc7b5-e751-4107-8fb6-5fe968e94f61 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:31.727911 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:31.746264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd26915-9cf7-4cce-b34b-8d4fb2b2f509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:31.747400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd26915-9cf7-4cce-b34b-8d4fb2b2f509 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:31.747937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd26915-9cf7-4cce-b34b-8d4fb2b2f509 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:31.748456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd26915-9cf7-4cce-b34b-8d4fb2b2f509 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:31.749423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd26915-9cf7-4cce-b34b-8d4fb2b2f509 None admin] RBAC: Adding query filter params (name=member) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:31.749976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd26915-9cf7-4cce-b34b-8d4fb2b2f509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:31.751595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd26915-9cf7-4cce-b34b-8d4fb2b2f509 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:31.761114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 207/423] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:00:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:31.767264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd26915-9cf7-4cce-b34b-8d4fb2b2f509 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:31.827968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 217/424] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:31.836059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca5969-1e37-4d0d-be7d-14adc7102884 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:31.918953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd26915-9cf7-4cce-b34b-8d4fb2b2f509 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:31.963385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:31.990690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92902fe1-b17e-4277-9932-21bce5753ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:31.991381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92902fe1-b17e-4277-9932-21bce5753ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:31.991737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92902fe1-b17e-4277-9932-21bce5753ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:31.992089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92902fe1-b17e-4277-9932-21bce5753ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:32.025962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca5969-1e37-4d0d-be7d-14adc7102884 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:32.062826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92902fe1-b17e-4277-9932-21bce5753ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:32.064544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92902fe1-b17e-4277-9932-21bce5753ff6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:32.091994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:32.108094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053833d-cd03-436f-806c-0756fccdf500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:32.108994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053833d-cd03-436f-806c-0756fccdf500 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:32.109285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053833d-cd03-436f-806c-0756fccdf500 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:32.109659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053833d-cd03-436f-806c-0756fccdf500 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:32.116096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053833d-cd03-436f-806c-0756fccdf500 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:32.117775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053833d-cd03-436f-806c-0756fccdf500 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:32.119846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 218/425] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:31 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:32.125567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053833d-cd03-436f-806c-0756fccdf500 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:32.170628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 208/426] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:00:32.176554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96fc23-7686-4486-bfa2-c04f20e3f3e4 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:32.176959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96fc23-7686-4486-bfa2-c04f20e3f3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:32.177348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96fc23-7686-4486-bfa2-c04f20e3f3e4 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:32.182601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 209/427] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:32.182601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96fc23-7686-4486-bfa2-c04f20e3f3e4 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:32.216200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:32.237305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053833d-cd03-436f-806c-0756fccdf500 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:32.272570 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79ac282-2e75-4012-a2c0-7a9b260fec4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:32.273977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ac282-2e75-4012-a2c0-7a9b260fec4e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:32.274425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ac282-2e75-4012-a2c0-7a9b260fec4e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:32.275207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79ac282-2e75-4012-a2c0-7a9b260fec4e None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:32.276798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ac282-2e75-4012-a2c0-7a9b260fec4e None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:32.279550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79ac282-2e75-4012-a2c0-7a9b260fec4e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:32.282049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 219/428] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:32 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:32.283438 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242c356-4b1e-47a3-8235-e38378ba0a45 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:32.283853 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242c356-4b1e-47a3-8235-e38378ba0a45 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:32.284257 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242c356-4b1e-47a3-8235-e38378ba0a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:32.303618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-a242c356-4b1e-47a3-8235-e38378ba0a45 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:32.303824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96fc23-7686-4486-bfa2-c04f20e3f3e4 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:32.305148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a242c356-4b1e-47a3-8235-e38378ba0a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:32.339554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d831f-1234-4bac-bdce-c682856cf03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:32.340533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d831f-1234-4bac-bdce-c682856cf03e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:32.341177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d831f-1234-4bac-bdce-c682856cf03e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:32.341470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d831f-1234-4bac-bdce-c682856cf03e None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:32.342782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d831f-1234-4bac-bdce-c682856cf03e None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:32.344408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d831f-1234-4bac-bdce-c682856cf03e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:32.350879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 210/429] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:32 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:32.351811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d831f-1234-4bac-bdce-c682856cf03e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:32.375313 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:32.388462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 220/430] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:32.389478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242c356-4b1e-47a3-8235-e38378ba0a45 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:32.447619 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:32.459653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d831f-1234-4bac-bdce-c682856cf03e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:32.495512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48875d06-b7cb-4ff1-9a67-98b6d5d86308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:32.496404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48875d06-b7cb-4ff1-9a67-98b6d5d86308 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:32.496784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48875d06-b7cb-4ff1-9a67-98b6d5d86308 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:32.497256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48875d06-b7cb-4ff1-9a67-98b6d5d86308 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:32.498203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48875d06-b7cb-4ff1-9a67-98b6d5d86308 None admin] RBAC: Adding query filter params (group.id=670d33c6c7f24ab1b3c3e3880dae2beb, role.id=6e32448ba01f41bcafb4cb0a839691fe, scope.project.id=a2c6e66812894558944eb8f6f156896d) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:32.498836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48875d06-b7cb-4ff1-9a67-98b6d5d86308 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:32.500250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48875d06-b7cb-4ff1-9a67-98b6d5d86308 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:32.507108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 211/431] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 5 15:00:32 2022] GET /identity/v3/role_assignments?group.id=670d33c6c7f24ab1b3c3e3880dae2beb&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:32.508090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48875d06-b7cb-4ff1-9a67-98b6d5d86308 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:32.516962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242c356-4b1e-47a3-8235-e38378ba0a45 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:32.555468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc685d1e-ed5a-4d38-8b1c-82432070898a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:32.556679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc685d1e-ed5a-4d38-8b1c-82432070898a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:32.557117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc685d1e-ed5a-4d38-8b1c-82432070898a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:32.557541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc685d1e-ed5a-4d38-8b1c-82432070898a None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:32.558959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc685d1e-ed5a-4d38-8b1c-82432070898a None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:32.560790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc685d1e-ed5a-4d38-8b1c-82432070898a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:32.567186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 221/432] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:32 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:32.569459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc685d1e-ed5a-4d38-8b1c-82432070898a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:32.644240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:32.685136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc685d1e-ed5a-4d38-8b1c-82432070898a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:32.690577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48875d06-b7cb-4ff1-9a67-98b6d5d86308 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:32.746153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c3b29-c1b8-44b4-a053-24fed0d669d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:32.748015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c3b29-c1b8-44b4-a053-24fed0d669d6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:32.749314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c3b29-c1b8-44b4-a053-24fed0d669d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:32.750377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744c3b29-c1b8-44b4-a053-24fed0d669d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:32.751635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c3b29-c1b8-44b4-a053-24fed0d669d6 None admin] RBAC: Adding query filter params (group.id=670d33c6c7f24ab1b3c3e3880dae2beb, role.id=6e32448ba01f41bcafb4cb0a839691fe, scope.project.id=96f8d6ac74154c299166a6e2bf295a72) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:32.752774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c3b29-c1b8-44b4-a053-24fed0d669d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:32.755928 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744c3b29-c1b8-44b4-a053-24fed0d669d6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:32.760069 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338c912-ca3d-4553-8847-7d72597f164f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:32.761175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338c912-ca3d-4553-8847-7d72597f164f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:32.761652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338c912-ca3d-4553-8847-7d72597f164f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:32.762150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338c912-ca3d-4553-8847-7d72597f164f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:32.779149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 222/433] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 5 15:00:32 2022] GET /identity/v3/role_assignments?group.id=670d33c6c7f24ab1b3c3e3880dae2beb&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=6e32448ba01f41bcafb4cb0a839691fe => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:32.792471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338c912-ca3d-4553-8847-7d72597f164f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-9338c912-ca3d-4553-8847-7d72597f164f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:32.801181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:32.802066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:32.802852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:32.813240 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 212/434] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:00:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:32.819789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c3b29-c1b8-44b4-a053-24fed0d669d6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:32.952503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744c3b29-c1b8-44b4-a053-24fed0d669d6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:32.993959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:33.001406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11747fdc-09c5-41c5-9439-72e8c5e1bf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:33.002083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11747fdc-09c5-41c5-9439-72e8c5e1bf74 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.002368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11747fdc-09c5-41c5-9439-72e8c5e1bf74 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.002647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11747fdc-09c5-41c5-9439-72e8c5e1bf74 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.003455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11747fdc-09c5-41c5-9439-72e8c5e1bf74 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:33.003762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11747fdc-09c5-41c5-9439-72e8c5e1bf74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:33.005211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11747fdc-09c5-41c5-9439-72e8c5e1bf74 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:33.014743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 223/435] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:33.019912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338c912-ca3d-4553-8847-7d72597f164f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:33.037807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:33.061206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc325a-d47a-47fe-bdf6-9fec26aaf301 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.061532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc325a-d47a-47fe-bdf6-9fec26aaf301 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.061778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc325a-d47a-47fe-bdf6-9fec26aaf301 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.063495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 224/436] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:00:33.070050 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3ff43-2f1e-4ac3-9afb-c6e70f2524f0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.070335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3ff43-2f1e-4ac3-9afb-c6e70f2524f0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.070613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3ff43-2f1e-4ac3-9afb-c6e70f2524f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.094431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-1ee3ff43-2f1e-4ac3-9afb-c6e70f2524f0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:33.097107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-1ee3ff43-2f1e-4ac3-9afb-c6e70f2524f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:33.155228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338c912-ca3d-4553-8847-7d72597f164f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:33.183128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 225/437] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:33.196808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c2be3-bf0f-4189-bed7-e6672fe370eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:33.197471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c2be3-bf0f-4189-bed7-e6672fe370eb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.197759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c2be3-bf0f-4189-bed7-e6672fe370eb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.198040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c2be3-bf0f-4189-bed7-e6672fe370eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.211311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3ff43-2f1e-4ac3-9afb-c6e70f2524f0 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:33.292178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c2be3-bf0f-4189-bed7-e6672fe370eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:33.294610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c2be3-bf0f-4189-bed7-e6672fe370eb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:33.311188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:33.344467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3ff43-2f1e-4ac3-9afb-c6e70f2524f0 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:33.367990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:33.383364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ccb72-4bf6-46d8-b1a3-689c899b4d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:33.384558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ccb72-4bf6-46d8-b1a3-689c899b4d52 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.384852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ccb72-4bf6-46d8-b1a3-689c899b4d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.385133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ccb72-4bf6-46d8-b1a3-689c899b4d52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.399055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ccb72-4bf6-46d8-b1a3-689c899b4d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:33.402149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 213/438] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:33.407543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c2be3-bf0f-4189-bed7-e6672fe370eb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-5c4ccb72-4bf6-46d8-b1a3-689c899b4d52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:33.407670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:33.408499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:33.409297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 226/439] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:00:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:33.414687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ccb72-4bf6-46d8-b1a3-689c899b4d52 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:33.509450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c2be3-bf0f-4189-bed7-e6672fe370eb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:33.515686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ccb72-4bf6-46d8-b1a3-689c899b4d52 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:33.523199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:33.545181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c557f-1c02-41cd-a0af-4089b2d61848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:33.546045 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c557f-1c02-41cd-a0af-4089b2d61848 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.546327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c557f-1c02-41cd-a0af-4089b2d61848 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.546599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c557f-1c02-41cd-a0af-4089b2d61848 None admin] PATH_INFO: `/v3/projects/a2c6e66812894558944eb8f6f156896d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.547782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c557f-1c02-41cd-a0af-4089b2d61848 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c6e66812894558944eb8f6f156896d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:33.549207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c557f-1c02-41cd-a0af-4089b2d61848 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:33.550542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e949a-b208-4040-9bf1-62d04e35a13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:33.550880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 214/440] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:33 2022] GET /identity/v3/projects/a2c6e66812894558944eb8f6f156896d => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:33.555890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e949a-b208-4040-9bf1-62d04e35a13b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.555890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e949a-b208-4040-9bf1-62d04e35a13b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.555890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e949a-b208-4040-9bf1-62d04e35a13b None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.556071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d3c25-3590-4947-bfb8-429cab1293fe None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.556071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d3c25-3590-4947-bfb8-429cab1293fe None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.556208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e949a-b208-4040-9bf1-62d04e35a13b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:33.556277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d3c25-3590-4947-bfb8-429cab1293fe None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.556341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e949a-b208-4040-9bf1-62d04e35a13b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:33.556404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 215/441] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:33.556467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e949a-b208-4040-9bf1-62d04e35a13b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:33.556972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d3c25-3590-4947-bfb8-429cab1293fe None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:33.562892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 227/442] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:33.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994af19-5591-4ec5-930b-4d8d9fafad90 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.564274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994af19-5591-4ec5-930b-4d8d9fafad90 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.564556 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994af19-5591-4ec5-930b-4d8d9fafad90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.580465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-2994af19-5591-4ec5-930b-4d8d9fafad90 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:33.581652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2994af19-5591-4ec5-930b-4d8d9fafad90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:33.620796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:33.647020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 228/443] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:33.647892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994af19-5591-4ec5-930b-4d8d9fafad90 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:33.660864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d3c25-3590-4947-bfb8-429cab1293fe None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:33.695775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d0da9-ebe5-4f92-a314-175f51fe7f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:33.696637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d0da9-ebe5-4f92-a314-175f51fe7f64 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.696927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d0da9-ebe5-4f92-a314-175f51fe7f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.697206 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d0da9-ebe5-4f92-a314-175f51fe7f64 None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.698413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d0da9-ebe5-4f92-a314-175f51fe7f64 None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:33.699881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d0da9-ebe5-4f92-a314-175f51fe7f64 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:33.701772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 216/444] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:33 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:33.702530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d0da9-ebe5-4f92-a314-175f51fe7f64 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:33.747511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994af19-5591-4ec5-930b-4d8d9fafad90 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:33.750931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:33.781243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf01db6-daba-4e14-9ffd-ddfd2cd8cef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:33.781887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf01db6-daba-4e14-9ffd-ddfd2cd8cef9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.782169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf01db6-daba-4e14-9ffd-ddfd2cd8cef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.782449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf01db6-daba-4e14-9ffd-ddfd2cd8cef9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.801979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d0da9-ebe5-4f92-a314-175f51fe7f64 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:33.822649 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:33.836109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4fba6-63eb-4dc2-b0fa-0a50706db89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:33.836881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4fba6-63eb-4dc2-b0fa-0a50706db89b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.837166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4fba6-63eb-4dc2-b0fa-0a50706db89b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.837439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4fba6-63eb-4dc2-b0fa-0a50706db89b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.847448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4fba6-63eb-4dc2-b0fa-0a50706db89b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:33.848010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf01db6-daba-4e14-9ffd-ddfd2cd8cef9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:33.849479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf01db6-daba-4e14-9ffd-ddfd2cd8cef9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8fa4fba6-63eb-4dc2-b0fa-0a50706db89b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:33.854915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:33.855902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 217/445] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:00:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:33.862643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4fba6-63eb-4dc2-b0fa-0a50706db89b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:33.945302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 229/446] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:33.946137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf01db6-daba-4e14-9ffd-ddfd2cd8cef9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:33.957324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4fba6-63eb-4dc2-b0fa-0a50706db89b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:33.967200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:33.991177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f809c-3620-44a1-a4b1-a31152d4f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:33.991817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f809c-3620-44a1-a4b1-a31152d4f46a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:33.992101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f809c-3620-44a1-a4b1-a31152d4f46a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:33.992375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f809c-3620-44a1-a4b1-a31152d4f46a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:33.993200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f809c-3620-44a1-a4b1-a31152d4f46a None admin] RBAC: Adding query filter params (group.id=670d33c6c7f24ab1b3c3e3880dae2beb, role.id=4301bced0259446cafb79118a53a015e, scope.project.id=a2c6e66812894558944eb8f6f156896d) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:33.993511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f809c-3620-44a1-a4b1-a31152d4f46a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:33.994947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f809c-3620-44a1-a4b1-a31152d4f46a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:34.000608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 218/447] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 5 15:00:33 2022] GET /identity/v3/role_assignments?group.id=670d33c6c7f24ab1b3c3e3880dae2beb&scope.project.id=a2c6e66812894558944eb8f6f156896d&role.id=4301bced0259446cafb79118a53a015e => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:34.001373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f809c-3620-44a1-a4b1-a31152d4f46a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:34.025838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:34.056718 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf01db6-daba-4e14-9ffd-ddfd2cd8cef9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:34.095921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017d53-e108-4c81-ad76-1f5186b90bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:34.097552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017d53-e108-4c81-ad76-1f5186b90bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:34.097984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017d53-e108-4c81-ad76-1f5186b90bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:34.098448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c017d53-e108-4c81-ad76-1f5186b90bb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:34.099246 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c017d53-e108-4c81-ad76-1f5186b90bb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:34.099561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c017d53-e108-4c81-ad76-1f5186b90bb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:34.100984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c017d53-e108-4c81-ad76-1f5186b90bb1 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:34.110432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 230/448] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:34.110740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f809c-3620-44a1-a4b1-a31152d4f46a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:34.116325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017d53-e108-4c81-ad76-1f5186b90bb1 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:34.151366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297be82-e690-44ab-8473-984cde13e98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:34.152531 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297be82-e690-44ab-8473-984cde13e98d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:34.153526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297be82-e690-44ab-8473-984cde13e98d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:34.153960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8297be82-e690-44ab-8473-984cde13e98d None admin] PATH_INFO: `/v3/projects/96f8d6ac74154c299166a6e2bf295a72` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:34.155210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297be82-e690-44ab-8473-984cde13e98d None admin] RBAC: Authorizing `identity:get_project(project_id=96f8d6ac74154c299166a6e2bf295a72)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:34.156654 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8297be82-e690-44ab-8473-984cde13e98d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:34.158504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 219/449] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:34 2022] GET /identity/v3/projects/96f8d6ac74154c299166a6e2bf295a72 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:34.163743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297be82-e690-44ab-8473-984cde13e98d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:34.215293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:34.234617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c017d53-e108-4c81-ad76-1f5186b90bb1 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:34.272401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54f1fa-d0ab-4d35-b590-0428ee794a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:34.273218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54f1fa-d0ab-4d35-b590-0428ee794a40 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:34.273612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54f1fa-d0ab-4d35-b590-0428ee794a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:34.273899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54f1fa-d0ab-4d35-b590-0428ee794a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:34.278933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:34.284430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8297be82-e690-44ab-8473-984cde13e98d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:34.325756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdc018-252b-44b2-b2e4-a20764a7b717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:34.327340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdc018-252b-44b2-b2e4-a20764a7b717 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:34.327672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdc018-252b-44b2-b2e4-a20764a7b717 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:34.329424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdc018-252b-44b2-b2e4-a20764a7b717 None admin] PATH_INFO: `/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:34.330956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdc018-252b-44b2-b2e4-a20764a7b717 None admin] RBAC: Authorizing `identity:get_group(group_id=670d33c6c7f24ab1b3c3e3880dae2beb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:34.332369 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdc018-252b-44b2-b2e4-a20764a7b717 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:34.335728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 220/450] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:34 2022] GET /identity/v3/groups/670d33c6c7f24ab1b3c3e3880dae2beb => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:34.339868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdc018-252b-44b2-b2e4-a20764a7b717 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:34.360929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54f1fa-d0ab-4d35-b590-0428ee794a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:34.362398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54f1fa-d0ab-4d35-b590-0428ee794a40 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:34.457241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdc018-252b-44b2-b2e4-a20764a7b717 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:34.468975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:34.485183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 231/451] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 05 15:00:34.490819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54f1fa-d0ab-4d35-b590-0428ee794a40 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:34.502838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b768997-f609-4c90-bce4-6d2411377b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:34.503744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b768997-f609-4c90-bce4-6d2411377b10 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:34.504171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b768997-f609-4c90-bce4-6d2411377b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:34.504590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b768997-f609-4c90-bce4-6d2411377b10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:34.505605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b768997-f609-4c90-bce4-6d2411377b10 None admin] RBAC: Adding query filter params (group.id=670d33c6c7f24ab1b3c3e3880dae2beb, role.id=4301bced0259446cafb79118a53a015e, scope.project.id=96f8d6ac74154c299166a6e2bf295a72) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:34.505918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b768997-f609-4c90-bce4-6d2411377b10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:34.507398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b768997-f609-4c90-bce4-6d2411377b10 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:34.514107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 221/452] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 5 15:00:34 2022] GET /identity/v3/role_assignments?group.id=670d33c6c7f24ab1b3c3e3880dae2beb&scope.project.id=96f8d6ac74154c299166a6e2bf295a72&role.id=4301bced0259446cafb79118a53a015e => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:34.528878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:34.599700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54f1fa-d0ab-4d35-b590-0428ee794a40 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:34.635553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77277aa4-48ec-448d-8daa-68785542aa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:34.636419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77277aa4-48ec-448d-8daa-68785542aa1f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:34.636701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77277aa4-48ec-448d-8daa-68785542aa1f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:34.636976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77277aa4-48ec-448d-8daa-68785542aa1f None admin] PATH_INFO: `/v3/projects/77588a8e554a4d32b7cfdcc698266c19` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:34.638257 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77277aa4-48ec-448d-8daa-68785542aa1f None admin] RBAC: Authorizing `identity:get_project(project_id=77588a8e554a4d32b7cfdcc698266c19)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:34.639746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77277aa4-48ec-448d-8daa-68785542aa1f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:34.641466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 232/453] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:00:34 2022] GET /identity/v3/projects/77588a8e554a4d32b7cfdcc698266c19 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:34.647182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b768997-f609-4c90-bce4-6d2411377b10 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:34.745585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b768997-f609-4c90-bce4-6d2411377b10 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:34.778711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af76acb-a16b-48cc-ab86-e38199541bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:34.779588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af76acb-a16b-48cc-ab86-e38199541bba None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:34.779866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af76acb-a16b-48cc-ab86-e38199541bba None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:34.780137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af76acb-a16b-48cc-ab86-e38199541bba None admin] PATH_INFO: `/v3/groups/c94e9ed8cc4f4b968d399dd168f1db29` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:34.781295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af76acb-a16b-48cc-ab86-e38199541bba None admin] RBAC: Authorizing `identity:get_group(group_id=c94e9ed8cc4f4b968d399dd168f1db29)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:34.782736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af76acb-a16b-48cc-ab86-e38199541bba None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:34.784363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 222/454] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 5 15:00:34 2022] GET /identity/v3/groups/c94e9ed8cc4f4b968d399dd168f1db29 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:34.790250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77277aa4-48ec-448d-8daa-68785542aa1f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:34.889921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77277aa4-48ec-448d-8daa-68785542aa1f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:34.892209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:34.924846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c8b65-dd74-41e3-b33b-aeb69315c5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:34.925502 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c8b65-dd74-41e3-b33b-aeb69315c5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:34.925790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c8b65-dd74-41e3-b33b-aeb69315c5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:34.926070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c8b65-dd74-41e3-b33b-aeb69315c5e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:34.926902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c8b65-dd74-41e3-b33b-aeb69315c5e4 None admin] RBAC: Adding query filter params (group.id=c94e9ed8cc4f4b968d399dd168f1db29, role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=77588a8e554a4d32b7cfdcc698266c19) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:34.927249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c8b65-dd74-41e3-b33b-aeb69315c5e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:34.928704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c8b65-dd74-41e3-b33b-aeb69315c5e4 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:34.934349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 233/455] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 5 15:00:34 2022] GET /identity/v3/role_assignments?group.id=c94e9ed8cc4f4b968d399dd168f1db29&scope.project.id=77588a8e554a4d32b7cfdcc698266c19&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:37.305792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8ad20-f0de-4fda-a591-163cb8470f28 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:37.306776 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8ad20-f0de-4fda-a591-163cb8470f28 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:37.307326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8ad20-f0de-4fda-a591-163cb8470f28 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:37.309703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 223/456] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:37.314448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4179f-b3e6-47ba-9b9c-1165285fb874 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:37.314855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4179f-b3e6-47ba-9b9c-1165285fb874 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:37.315019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4179f-b3e6-47ba-9b9c-1165285fb874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:37.341109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-97a4179f-b3e6-47ba-9b9c-1165285fb874 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:37.342539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-97a4179f-b3e6-47ba-9b9c-1165285fb874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:37.438634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 234/457] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:37.511110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8ad20-f0de-4fda-a591-163cb8470f28 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:37.538515 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e2dc7-eea8-4110-9d7a-89e31b2a0fd2 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:37.539016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e2dc7-eea8-4110-9d7a-89e31b2a0fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:37.539540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e2dc7-eea8-4110-9d7a-89e31b2a0fd2 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:37.542358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 235/458] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:37.549548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705ac8-31e4-4c30-a7a3-2ff60feab956 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:37.550295 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705ac8-31e4-4c30-a7a3-2ff60feab956 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:37.550982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705ac8-31e4-4c30-a7a3-2ff60feab956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:37.565784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:37.589969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-64705ac8-31e4-4c30-a7a3-2ff60feab956 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:37.591260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-64705ac8-31e4-4c30-a7a3-2ff60feab956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:37.669391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8ad20-f0de-4fda-a591-163cb8470f28 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:37.712629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 236/459] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:37.732174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8f3e5-d129-402e-ba9b-aeeca3108088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:37.732972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8f3e5-d129-402e-ba9b-aeeca3108088 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:37.733255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8f3e5-d129-402e-ba9b-aeeca3108088 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:37.733527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8f3e5-d129-402e-ba9b-aeeca3108088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:37.742376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64705ac8-31e4-4c30-a7a3-2ff60feab956 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:37.763420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8f3e5-d129-402e-ba9b-aeeca3108088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:37.764816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8f3e5-d129-402e-ba9b-aeeca3108088 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:37.777775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d1f8f3e5-d129-402e-ba9b-aeeca3108088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:37.791931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:37.792835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:37.794212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:37.794212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:37.794212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:37.794212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:37.796902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 224/460] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:37.805328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8f3e5-d129-402e-ba9b-aeeca3108088 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:37.866574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64705ac8-31e4-4c30-a7a3-2ff60feab956 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:37.902193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9afc82-0a86-43f7-a3e9-e5c7483c39ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:37.903708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9afc82-0a86-43f7-a3e9-e5c7483c39ff None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:37.904108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9afc82-0a86-43f7-a3e9-e5c7483c39ff None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:37.904444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9afc82-0a86-43f7-a3e9-e5c7483c39ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:37.917191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9afc82-0a86-43f7-a3e9-e5c7483c39ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:37.919508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9afc82-0a86-43f7-a3e9-e5c7483c39ff None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:37.924390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:37.932996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8f3e5-d129-402e-ba9b-aeeca3108088 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-6d9afc82-0a86-43f7-a3e9-e5c7483c39ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:37.943594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:37.944462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:37.945737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:37.945737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:37.945737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:37.945737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:37.945737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 237/461] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:37.946377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09618c4b-6184-4e57-ba68-2e3f12d7c73f None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:37.946672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09618c4b-6184-4e57-ba68-2e3f12d7c73f None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:37.946950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09618c4b-6184-4e57-ba68-2e3f12d7c73f None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:37.952563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 238/462] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:37.953307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09618c4b-6184-4e57-ba68-2e3f12d7c73f None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:37.980453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf989c6e-fd42-4a29-886a-904eb38d52cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:37.982658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf989c6e-fd42-4a29-886a-904eb38d52cb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:37.983427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf989c6e-fd42-4a29-886a-904eb38d52cb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:37.983907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf989c6e-fd42-4a29-886a-904eb38d52cb None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:37.985544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf989c6e-fd42-4a29-886a-904eb38d52cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:37.986194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf989c6e-fd42-4a29-886a-904eb38d52cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:37.988176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf989c6e-fd42-4a29-886a-904eb38d52cb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:38.005191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 225/463] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:38.006778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ab06a-cbbc-4fff-baad-99ebe2e2dc5e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:38.007276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ab06a-cbbc-4fff-baad-99ebe2e2dc5e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:38.007714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ab06a-cbbc-4fff-baad-99ebe2e2dc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:38.031228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-064ab06a-cbbc-4fff-baad-99ebe2e2dc5e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:38.032754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-064ab06a-cbbc-4fff-baad-99ebe2e2dc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:38.057330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:38.086016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09618c4b-6184-4e57-ba68-2e3f12d7c73f None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:38.115497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 226/464] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:38.116608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ab06a-cbbc-4fff-baad-99ebe2e2dc5e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:38.128754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f0e09-4d55-4d27-8d73-bc895b2149e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:38.129728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f0e09-4d55-4d27-8d73-bc895b2149e7 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:38.130222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f0e09-4d55-4d27-8d73-bc895b2149e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:38.130705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f0e09-4d55-4d27-8d73-bc895b2149e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:38.131691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f0e09-4d55-4d27-8d73-bc895b2149e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:38.132207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f0e09-4d55-4d27-8d73-bc895b2149e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:38.134679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5f0e09-4d55-4d27-8d73-bc895b2149e7 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:38.148849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 239/465] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:38.154973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f0e09-4d55-4d27-8d73-bc895b2149e7 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:38.253514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ab06a-cbbc-4fff-baad-99ebe2e2dc5e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:38.271633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:38.286775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f0e09-4d55-4d27-8d73-bc895b2149e7 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:38.295903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1b7ca-7e59-41aa-b0cb-91337f00e385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:38.296971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1b7ca-7e59-41aa-b0cb-91337f00e385 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:38.297419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1b7ca-7e59-41aa-b0cb-91337f00e385 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:38.297844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1b7ca-7e59-41aa-b0cb-91337f00e385 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:38.298696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1b7ca-7e59-41aa-b0cb-91337f00e385 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:38.300400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1b7ca-7e59-41aa-b0cb-91337f00e385 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:38.331104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c653908-cf81-4cd7-af10-5081f0f5a9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:38.331104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653908-cf81-4cd7-af10-5081f0f5a9a9 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:38.331104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653908-cf81-4cd7-af10-5081f0f5a9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:38.331104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653908-cf81-4cd7-af10-5081f0f5a9a9 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:38.331886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c653908-cf81-4cd7-af10-5081f0f5a9a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:38.331886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c653908-cf81-4cd7-af10-5081f0f5a9a9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:38.332517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 227/466] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:00:38 2022] POST /identity/v3/users => generated 246 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:00:38.334003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1b7ca-7e59-41aa-b0cb-91337f00e385 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:38.342519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:38.356838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 240/467] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:00:38 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:00:38.358685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055557cf-e2e1-47f7-8e38-f9f9f22fa201 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:38.359383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055557cf-e2e1-47f7-8e38-f9f9f22fa201 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:38.359745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055557cf-e2e1-47f7-8e38-f9f9f22fa201 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:38.363098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 241/468] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:38.367769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ea3df-9d7c-43e3-9334-8d63e2c84ffc None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:38.368494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ea3df-9d7c-43e3-9334-8d63e2c84ffc None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:38.369175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7ea3df-9d7c-43e3-9334-8d63e2c84ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:38.404625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4d7ea3df-9d7c-43e3-9334-8d63e2c84ffc None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:38.410651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4d7ea3df-9d7c-43e3-9334-8d63e2c84ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:38.540666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1b7ca-7e59-41aa-b0cb-91337f00e385 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:38.550093 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 242/469] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:38.581435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ea3df-9d7c-43e3-9334-8d63e2c84ffc None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:38.584589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3861c6-abd1-4863-bce5-72f744d38b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:38.585467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3861c6-abd1-4863-bce5-72f744d38b1d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:38.585828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3861c6-abd1-4863-bce5-72f744d38b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:38.586172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3861c6-abd1-4863-bce5-72f744d38b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:38.603950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:38.608206 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3861c6-abd1-4863-bce5-72f744d38b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:38.612817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3861c6-abd1-4863-bce5-72f744d38b1d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4f3861c6-abd1-4863-bce5-72f744d38b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:38.644288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:38.645198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:38.646025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:38.646025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:38.646025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:38.646025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:38.646025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 228/470] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:38.652638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3861c6-abd1-4863-bce5-72f744d38b1d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:38.734789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7ea3df-9d7c-43e3-9334-8d63e2c84ffc None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:38.756915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3861c6-abd1-4863-bce5-72f744d38b1d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:38.768448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:38.789992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fa8e7-8001-4956-bee8-fb7b0738aeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:38.791242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fa8e7-8001-4956-bee8-fb7b0738aeec None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:38.793378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fa8e7-8001-4956-bee8-fb7b0738aeec None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:38.793974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957fa8e7-8001-4956-bee8-fb7b0738aeec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:38.815977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fa8e7-8001-4956-bee8-fb7b0738aeec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:38.818240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957fa8e7-8001-4956-bee8-fb7b0738aeec None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:38.820401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19dffa-78aa-467e-9b89-8e0cd89cbfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:38.821299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19dffa-78aa-467e-9b89-8e0cd89cbfee None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:38.821942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19dffa-78aa-467e-9b89-8e0cd89cbfee None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:38.823506 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19dffa-78aa-467e-9b89-8e0cd89cbfee None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:38.824466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19dffa-78aa-467e-9b89-8e0cd89cbfee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:38.825284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19dffa-78aa-467e-9b89-8e0cd89cbfee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:38.827795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de19dffa-78aa-467e-9b89-8e0cd89cbfee None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-957fa8e7-8001-4956-bee8-fb7b0738aeec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:38.852406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:38.853328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:38.854172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:38.854172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:38.854172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:38.854172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:38.858998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 229/471] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:38.863172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 243/472] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:38.868811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fa8e7-8001-4956-bee8-fb7b0738aeec None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:38.869391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19dffa-78aa-467e-9b89-8e0cd89cbfee None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:38.943974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:38.998364 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de19dffa-78aa-467e-9b89-8e0cd89cbfee None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:39.031998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957fa8e7-8001-4956-bee8-fb7b0738aeec None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:39.034256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b409b57a-925d-4e23-9c85-2d564208fa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:39.034935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409b57a-925d-4e23-9c85-2d564208fa02 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:39.035253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409b57a-925d-4e23-9c85-2d564208fa02 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:39.035533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b409b57a-925d-4e23-9c85-2d564208fa02 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:39.036264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409b57a-925d-4e23-9c85-2d564208fa02 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:39.037715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b409b57a-925d-4e23-9c85-2d564208fa02 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:39.065909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 230/473] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:00:38 2022] POST /identity/v3/users => generated 248 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:00:39.077745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab30a4e-d7d8-4084-b56c-d44c1d2b0a8b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:39.078392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab30a4e-d7d8-4084-b56c-d44c1d2b0a8b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:39.078869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab30a4e-d7d8-4084-b56c-d44c1d2b0a8b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:39.080970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 231/474] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:39 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:39.087010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da9b9b-7a5b-4d48-a631-ca47014cbe95 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:39.087698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da9b9b-7a5b-4d48-a631-ca47014cbe95 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:39.088352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da9b9b-7a5b-4d48-a631-ca47014cbe95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:39.102742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462613c-2f38-4000-ab59-2cba77c28d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:39.103498 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462613c-2f38-4000-ab59-2cba77c28d34 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:39.103786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462613c-2f38-4000-ab59-2cba77c28d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:39.104094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462613c-2f38-4000-ab59-2cba77c28d34 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:39.104828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462613c-2f38-4000-ab59-2cba77c28d34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:39.105140 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462613c-2f38-4000-ab59-2cba77c28d34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:39.107271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462613c-2f38-4000-ab59-2cba77c28d34 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:39.110320 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-23da9b9b-7a5b-4d48-a631-ca47014cbe95 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:39.111972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-23da9b9b-7a5b-4d48-a631-ca47014cbe95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:39.142995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 244/475] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:39.150393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462613c-2f38-4000-ab59-2cba77c28d34 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:39.203635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 232/476] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:39.235027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:39.238429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da9b9b-7a5b-4d48-a631-ca47014cbe95 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:39.285448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:39.312769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462613c-2f38-4000-ab59-2cba77c28d34 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:39.355146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23da9b9b-7a5b-4d48-a631-ca47014cbe95 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:39.359111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd786a0b-6dfd-47e4-a60e-2b1e72ffa194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:39.359796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd786a0b-6dfd-47e4-a60e-2b1e72ffa194 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:39.360087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd786a0b-6dfd-47e4-a60e-2b1e72ffa194 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:39.360371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd786a0b-6dfd-47e4-a60e-2b1e72ffa194 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:39.361104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd786a0b-6dfd-47e4-a60e-2b1e72ffa194 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:39.362553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd786a0b-6dfd-47e4-a60e-2b1e72ffa194 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:39.389408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 245/477] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:00:39 2022] POST /identity/v3/users => generated 249 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:00:39.401133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30156be8-f179-45ea-a8b0-c7965b42df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:39.401830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30156be8-f179-45ea-a8b0-c7965b42df1c None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:39.402112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30156be8-f179-45ea-a8b0-c7965b42df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:39.402393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30156be8-f179-45ea-a8b0-c7965b42df1c None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:39.404168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30156be8-f179-45ea-a8b0-c7965b42df1c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:39.405782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30156be8-f179-45ea-a8b0-c7965b42df1c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:39.413617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.sql.core [None req-30156be8-f179-45ea-a8b0-c7965b42df1c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 05 15:00:39.413617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 05 15:00:39.413617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [parameters: {'id': 'd6bb7779b0d6409c972ed1880fd267ec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 05 15:00:39.413617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80352) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-30156be8-f179-45ea-a8b0-c7965b42df1c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application result.read() Jul 05 15:00:39.434662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return ref.to_dict() Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:00:39.435627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application yield resource Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self.session.rollback() Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self.force_reraise() Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise self.value Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application session.commit() Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 05 15:00:39.436527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._prepare_impl() Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self.session.flush() Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._flush(objects) Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application compat.raise_( Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application flush_context.execute() Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rec.execute(self) Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 05 15:00:39.437415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:00:39.438284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application result.read() Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:00:39.439156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [parameters: {'id': 'd6bb7779b0d6409c972ed1880fd267ec', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:39.439986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 05 15:00:39.440810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:39.441815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 233/478] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jul 5 15:00:39 2022] POST /identity/v3/roles => generated 144 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:39.443022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd786a0b-6dfd-47e4-a60e-2b1e72ffa194 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:39.557128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:39.564772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd786a0b-6dfd-47e4-a60e-2b1e72ffa194 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:39.601259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d2e1d-8471-4127-9661-c2cf3c1ceede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:39.602077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d2e1d-8471-4127-9661-c2cf3c1ceede None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:39.602360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d2e1d-8471-4127-9661-c2cf3c1ceede None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:39.602634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d2e1d-8471-4127-9661-c2cf3c1ceede None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:39.613442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d2e1d-8471-4127-9661-c2cf3c1ceede None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-4c6d2e1d-8471-4127-9661-c2cf3c1ceede None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:39.622064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:39.622935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:39.623777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 246/479] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:00:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:39.629210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30156be8-f179-45ea-a8b0-c7965b42df1c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:39.739012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30156be8-f179-45ea-a8b0-c7965b42df1c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:39.740560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:39.780028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf2799-2d19-47c3-ad21-df1d72c5abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:39.780708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf2799-2d19-47c3-ad21-df1d72c5abf9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:39.780992 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf2799-2d19-47c3-ad21-df1d72c5abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:39.781267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf2799-2d19-47c3-ad21-df1d72c5abf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:39.782023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf2799-2d19-47c3-ad21-df1d72c5abf9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:39.782333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf2799-2d19-47c3-ad21-df1d72c5abf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:39.783828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf2799-2d19-47c3-ad21-df1d72c5abf9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:39.792801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 234/480] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:40.717328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84b72-544c-40f3-afc6-76b44b4679c6 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:40.717874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84b72-544c-40f3-afc6-76b44b4679c6 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:40.717874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb84b72-544c-40f3-afc6-76b44b4679c6 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:40.719611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 247/481] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:40.724629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd590489-08d4-422d-bc4d-764ce1388a5e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:40.725151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd590489-08d4-422d-bc4d-764ce1388a5e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:40.725318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd590489-08d4-422d-bc4d-764ce1388a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:40.748378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fd590489-08d4-422d-bc4d-764ce1388a5e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:40.749570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fd590489-08d4-422d-bc4d-764ce1388a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:40.839226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 235/482] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:40.876508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb84b72-544c-40f3-afc6-76b44b4679c6 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:40.900767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7d944-e709-432b-acaa-faf2b1c0e3bd None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:40.901055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7d944-e709-432b-acaa-faf2b1c0e3bd None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:40.901384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7d944-e709-432b-acaa-faf2b1c0e3bd None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:40.904292 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 236/483] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:00:40.909818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a316b3e-4816-42c5-b831-05454cd915a8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:40.910107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a316b3e-4816-42c5-b831-05454cd915a8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:40.910384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a316b3e-4816-42c5-b831-05454cd915a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:40.932784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8a316b3e-4816-42c5-b831-05454cd915a8 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:40.934495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8a316b3e-4816-42c5-b831-05454cd915a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:41.019980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 237/484] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:41.043254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a316b3e-4816-42c5-b831-05454cd915a8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:41.068724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb84b72-544c-40f3-afc6-76b44b4679c6 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:41.069199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:41.144519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeeffce-9250-4f24-baf5-752766d2121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:41.145424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeeffce-9250-4f24-baf5-752766d2121a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:41.145712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeeffce-9250-4f24-baf5-752766d2121a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:41.145987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeeffce-9250-4f24-baf5-752766d2121a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:41.148550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a316b3e-4816-42c5-b831-05454cd915a8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:41.169469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeeffce-9250-4f24-baf5-752766d2121a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-bdeeffce-9250-4f24-baf5-752766d2121a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:41.181458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:41.182397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:41.183235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 248/485] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:41.187801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee94371-cdd1-4b83-866e-35d323ee35f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:41.188849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee94371-cdd1-4b83-866e-35d323ee35f7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:41.189132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeeffce-9250-4f24-baf5-752766d2121a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:41.189636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee94371-cdd1-4b83-866e-35d323ee35f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:41.190106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee94371-cdd1-4b83-866e-35d323ee35f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:41.202797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee94371-cdd1-4b83-866e-35d323ee35f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8ee94371-cdd1-4b83-866e-35d323ee35f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:41.210712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:41.211636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:41.212426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:41.215065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 238/486] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:41.220937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee94371-cdd1-4b83-866e-35d323ee35f7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:41.227864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:41.308453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:41.325483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee94371-cdd1-4b83-866e-35d323ee35f7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:41.359679 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93623b1a-f24a-49da-b7e0-e15745239ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:41.360705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93623b1a-f24a-49da-b7e0-e15745239ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:41.361188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93623b1a-f24a-49da-b7e0-e15745239ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:41.361667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93623b1a-f24a-49da-b7e0-e15745239ea8 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:41.362603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93623b1a-f24a-49da-b7e0-e15745239ea8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:41.363148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93623b1a-f24a-49da-b7e0-e15745239ea8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:41.364721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93623b1a-f24a-49da-b7e0-e15745239ea8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:41.373445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 239/487] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:41.378999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93623b1a-f24a-49da-b7e0-e15745239ea8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:41.403492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeeffce-9250-4f24-baf5-752766d2121a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:41.477121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bd861-22cd-4a2c-a0a1-a66b31c1f257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:41.477801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bd861-22cd-4a2c-a0a1-a66b31c1f257 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:41.478141 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bd861-22cd-4a2c-a0a1-a66b31c1f257 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:41.478371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bd861-22cd-4a2c-a0a1-a66b31c1f257 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:41.479171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051bd861-22cd-4a2c-a0a1-a66b31c1f257 None admin] RBAC: Adding query filter params (name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:41.479615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051bd861-22cd-4a2c-a0a1-a66b31c1f257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:41.480927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051bd861-22cd-4a2c-a0a1-a66b31c1f257 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:41.482671 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93623b1a-f24a-49da-b7e0-e15745239ea8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:41.497922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 249/488] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:41.503418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bd861-22cd-4a2c-a0a1-a66b31c1f257 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:41.517603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40110d-8ac6-4734-a048-4c11904f614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:41.518708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40110d-8ac6-4734-a048-4c11904f614c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:41.519221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40110d-8ac6-4734-a048-4c11904f614c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:41.519693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40110d-8ac6-4734-a048-4c11904f614c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:41.531484 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40110d-8ac6-4734-a048-4c11904f614c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:41.533273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40110d-8ac6-4734-a048-4c11904f614c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:41.540116 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-bb40110d-8ac6-4734-a048-4c11904f614c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:41.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:41.552458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:41.553526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:41.553526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:41.553526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:41.553526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:41.555083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 240/489] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:41.560516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40110d-8ac6-4734-a048-4c11904f614c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:41.657683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:41.664729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40110d-8ac6-4734-a048-4c11904f614c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:41.699240 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eecdf2-181f-4ada-b8ef-9b3314ad779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:41.700159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eecdf2-181f-4ada-b8ef-9b3314ad779d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:41.700542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eecdf2-181f-4ada-b8ef-9b3314ad779d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:41.700919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eecdf2-181f-4ada-b8ef-9b3314ad779d None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:41.702018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eecdf2-181f-4ada-b8ef-9b3314ad779d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:41.702299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bd861-22cd-4a2c-a0a1-a66b31c1f257 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:41.702506 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eecdf2-181f-4ada-b8ef-9b3314ad779d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:41.704053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eecdf2-181f-4ada-b8ef-9b3314ad779d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:41.717407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 241/490] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:41.722712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eecdf2-181f-4ada-b8ef-9b3314ad779d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:41.774694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52475c-be9e-4d11-8552-87b53c04d1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:41.775566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52475c-be9e-4d11-8552-87b53c04d1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:41.775903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52475c-be9e-4d11-8552-87b53c04d1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:41.776183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52475c-be9e-4d11-8552-87b53c04d1a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:41.793129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:41.800239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52475c-be9e-4d11-8552-87b53c04d1a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:41.801736 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52475c-be9e-4d11-8552-87b53c04d1a8 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-fb52475c-be9e-4d11-8552-87b53c04d1a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:41.832329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:41.833208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:41.834094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:41.834094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:41.834094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:41.834094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:41.837430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eecdf2-181f-4ada-b8ef-9b3314ad779d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:41.837974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 250/491] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:41.839195 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52475c-be9e-4d11-8552-87b53c04d1a8 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:41.873816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fdd02-ed1d-4721-85db-340f425e00be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:41.874722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fdd02-ed1d-4721-85db-340f425e00be None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:41.875157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fdd02-ed1d-4721-85db-340f425e00be None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:41.875443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fdd02-ed1d-4721-85db-340f425e00be None admin] PATH_INFO: `/v3/users/nova` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:41.883565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fdd02-ed1d-4721-85db-340f425e00be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:41.885113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fdd02-ed1d-4721-85db-340f425e00be None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c21fdd02-ed1d-4721-85db-340f425e00be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:41.901387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:41.902353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:41.903803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 242/492] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 5 15:00:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:41.905066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966ca72-4775-465f-93f6-736b8826a313 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:41.905066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966ca72-4775-465f-93f6-736b8826a313 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:41.905856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2966ca72-4775-465f-93f6-736b8826a313 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:41.911254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 243/493] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:41.912140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966ca72-4775-465f-93f6-736b8826a313 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:41.957306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52475c-be9e-4d11-8552-87b53c04d1a8 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:41.989600 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:41.999559 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bc6c8-cdb7-4cda-8d8b-80660fd72c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:42.000001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bc6c8-cdb7-4cda-8d8b-80660fd72c12 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.000288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bc6c8-cdb7-4cda-8d8b-80660fd72c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.000663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bc6c8-cdb7-4cda-8d8b-80660fd72c12 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.001402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bc6c8-cdb7-4cda-8d8b-80660fd72c12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:42.001713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bc6c8-cdb7-4cda-8d8b-80660fd72c12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:42.003102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89bc6c8-cdb7-4cda-8d8b-80660fd72c12 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:42.015676 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 251/494] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:42.017056 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59c13-4a62-4f00-81ca-222ae2340ff9 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.017293 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59c13-4a62-4f00-81ca-222ae2340ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.017577 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a59c13-4a62-4f00-81ca-222ae2340ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.031293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2966ca72-4775-465f-93f6-736b8826a313 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:42.034541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-75a59c13-4a62-4f00-81ca-222ae2340ff9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:42.035553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-75a59c13-4a62-4f00-81ca-222ae2340ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:42.062312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:42.078158 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b887aa-ce95-4a25-8577-d4d3bb30ee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:42.079260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b887aa-ce95-4a25-8577-d4d3bb30ee9b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.080257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b887aa-ce95-4a25-8577-d4d3bb30ee9b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.080745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b887aa-ce95-4a25-8577-d4d3bb30ee9b None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.082259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b887aa-ce95-4a25-8577-d4d3bb30ee9b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:42.082946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b887aa-ce95-4a25-8577-d4d3bb30ee9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:42.085280 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b887aa-ce95-4a25-8577-d4d3bb30ee9b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:42.119884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 252/495] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:42.122966 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a59c13-4a62-4f00-81ca-222ae2340ff9 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:42.146014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 244/496] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 5 15:00:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:42.146869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b887aa-ce95-4a25-8577-d4d3bb30ee9b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:42.242988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a59c13-4a62-4f00-81ca-222ae2340ff9 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:42.260048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b887aa-ce95-4a25-8577-d4d3bb30ee9b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:42.261741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:42.281205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3dbee-e688-46bd-a616-2837dfa972a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:42.282465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3dbee-e688-46bd-a616-2837dfa972a0 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.282924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3dbee-e688-46bd-a616-2837dfa972a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.283241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3dbee-e688-46bd-a616-2837dfa972a0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.290602 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3dbee-e688-46bd-a616-2837dfa972a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:42.292095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3dbee-e688-46bd-a616-2837dfa972a0 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:42.296577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ab65a-0a80-4b10-a1ea-94a22414a913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:42.297367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ab65a-0a80-4b10-a1ea-94a22414a913 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.297644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ab65a-0a80-4b10-a1ea-94a22414a913 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.297916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ab65a-0a80-4b10-a1ea-94a22414a913 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.308119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ab65a-0a80-4b10-a1ea-94a22414a913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-53c3dbee-e688-46bd-a616-2837dfa972a0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:42.310120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:42.310994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:42.312196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 253/497] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:42.313103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb732be-285c-49e0-9394-176a8757b645 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.313258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb732be-285c-49e0-9394-176a8757b645 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.313518 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb732be-285c-49e0-9394-176a8757b645 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.315020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 254/498] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:42.315756 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb732be-285c-49e0-9394-176a8757b645 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-680ab65a-0a80-4b10-a1ea-94a22414a913 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:42.320985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:42.321899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:42.322698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:42.327509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 245/499] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:42.328492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ab65a-0a80-4b10-a1ea-94a22414a913 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:42.366608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:42.432203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb732be-285c-49e0-9394-176a8757b645 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:42.437483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ab65a-0a80-4b10-a1ea-94a22414a913 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:42.472965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace185a-32c0-4624-b3f2-07180feac723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:42.473620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace185a-32c0-4624-b3f2-07180feac723 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.473913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace185a-32c0-4624-b3f2-07180feac723 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.474229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace185a-32c0-4624-b3f2-07180feac723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.486518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d44995-21e6-41b1-9945-670880fd8039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:42.487226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d44995-21e6-41b1-9945-670880fd8039 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.487519 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d44995-21e6-41b1-9945-670880fd8039 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.487801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d44995-21e6-41b1-9945-670880fd8039 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.488665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d44995-21e6-41b1-9945-670880fd8039 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:42.488984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d44995-21e6-41b1-9945-670880fd8039 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:42.490446 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d44995-21e6-41b1-9945-670880fd8039 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:42.508880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 246/500] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 5 15:00:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:42.510232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902edd6-91b3-46e9-b629-b62364720a34 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.510525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902edd6-91b3-46e9-b629-b62364720a34 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.510812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8902edd6-91b3-46e9-b629-b62364720a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.526711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:42.529381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8902edd6-91b3-46e9-b629-b62364720a34 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:42.531406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8902edd6-91b3-46e9-b629-b62364720a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:42.548910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace185a-32c0-4624-b3f2-07180feac723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:42.550401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace185a-32c0-4624-b3f2-07180feac723 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:42.615068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 247/501] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:42.615068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902edd6-91b3-46e9-b629-b62364720a34 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:42.641935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:42.673319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 255/502] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:42.674463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace185a-32c0-4624-b3f2-07180feac723 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:42.727398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8902edd6-91b3-46e9-b629-b62364720a34 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:42.751799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:42.765886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9480c6-0714-4794-ab54-f8ef05b025ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:42.766565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9480c6-0714-4794-ab54-f8ef05b025ce None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.766855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9480c6-0714-4794-ab54-f8ef05b025ce None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.767173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9480c6-0714-4794-ab54-f8ef05b025ce None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.767935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9480c6-0714-4794-ab54-f8ef05b025ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:42.768250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9480c6-0714-4794-ab54-f8ef05b025ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:42.769641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9480c6-0714-4794-ab54-f8ef05b025ce None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:42.774731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace185a-32c0-4624-b3f2-07180feac723 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:42.780692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 248/503] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:42.781423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9480c6-0714-4794-ab54-f8ef05b025ce None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:42.812451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7f2caf-bb35-426a-8102-08a0fb570042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:42.813105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f2caf-bb35-426a-8102-08a0fb570042 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.813390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f2caf-bb35-426a-8102-08a0fb570042 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.813668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7f2caf-bb35-426a-8102-08a0fb570042 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.864473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:42.879807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9480c6-0714-4794-ab54-f8ef05b025ce None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:42.880031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f2caf-bb35-426a-8102-08a0fb570042 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:42.881502 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7f2caf-bb35-426a-8102-08a0fb570042 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:42.914805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b11d8-0833-4638-8454-cf86907525f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:42.915564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b11d8-0833-4638-8454-cf86907525f7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.915850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b11d8-0833-4638-8454-cf86907525f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.916127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b11d8-0833-4638-8454-cf86907525f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.926197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b11d8-0833-4638-8454-cf86907525f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-3b5b11d8-0833-4638-8454-cf86907525f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:42.933840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:42.934816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:42.935893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 249/504] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:42.936756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b11d8-0833-4638-8454-cf86907525f7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:42.961277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:42.983731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 256/505] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:42.985038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5eb2f-6ac6-4acb-9279-66083b158d14 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:42.985325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5eb2f-6ac6-4acb-9279-66083b158d14 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:42.985599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c5eb2f-6ac6-4acb-9279-66083b158d14 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:42.986879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 257/506] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:42.987603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5eb2f-6ac6-4acb-9279-66083b158d14 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.042271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b11d8-0833-4638-8454-cf86907525f7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.077269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05bc71-84ad-4b3c-8cad-f64f9f22c07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.078064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05bc71-84ad-4b3c-8cad-f64f9f22c07a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.078353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05bc71-84ad-4b3c-8cad-f64f9f22c07a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.078632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05bc71-84ad-4b3c-8cad-f64f9f22c07a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.086897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c5eb2f-6ac6-4acb-9279-66083b158d14 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.089084 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05bc71-84ad-4b3c-8cad-f64f9f22c07a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.091186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05bc71-84ad-4b3c-8cad-f64f9f22c07a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.097449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-aa05bc71-84ad-4b3c-8cad-f64f9f22c07a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:43.109166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:43.110071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:43.110959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:43.110959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:43.110959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:43.110959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:43.110959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 250/507] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:43.111270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05bc71-84ad-4b3c-8cad-f64f9f22c07a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.124269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86605a0f-cdd2-4eb2-af13-be190808b5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.125059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86605a0f-cdd2-4eb2-af13-be190808b5ea None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.125346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86605a0f-cdd2-4eb2-af13-be190808b5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.125627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86605a0f-cdd2-4eb2-af13-be190808b5ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.135665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86605a0f-cdd2-4eb2-af13-be190808b5ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.137682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86605a0f-cdd2-4eb2-af13-be190808b5ea None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-86605a0f-cdd2-4eb2-af13-be190808b5ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:43.155354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:43.156205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:43.157066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:43.157066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:43.157066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:43.157066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:43.157066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 258/508] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:43.157588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86605a0f-cdd2-4eb2-af13-be190808b5ea None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.186615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:43.217781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05bc71-84ad-4b3c-8cad-f64f9f22c07a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.250924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc53bc-eb75-4cd0-8326-ecce9a4cb284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.251614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc53bc-eb75-4cd0-8326-ecce9a4cb284 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.251897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc53bc-eb75-4cd0-8326-ecce9a4cb284 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.252178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc53bc-eb75-4cd0-8326-ecce9a4cb284 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.252919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc53bc-eb75-4cd0-8326-ecce9a4cb284 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:43.253228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc53bc-eb75-4cd0-8326-ecce9a4cb284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.254604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc53bc-eb75-4cd0-8326-ecce9a4cb284 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.255531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86605a0f-cdd2-4eb2-af13-be190808b5ea None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.266832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 251/509] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:43.266832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7e58-5544-4725-b725-2cda29a8ccc3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.266832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7e58-5544-4725-b725-2cda29a8ccc3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.266832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7e58-5544-4725-b725-2cda29a8ccc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.280477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-86bc7e58-5544-4725-b725-2cda29a8ccc3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:43.281685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-86bc7e58-5544-4725-b725-2cda29a8ccc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:43.294288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17f6ca-d8ab-4657-aaa1-2214456ba24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.295092 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17f6ca-d8ab-4657-aaa1-2214456ba24c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.295374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17f6ca-d8ab-4657-aaa1-2214456ba24c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.295899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17f6ca-d8ab-4657-aaa1-2214456ba24c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.305363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17f6ca-d8ab-4657-aaa1-2214456ba24c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.306801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17f6ca-d8ab-4657-aaa1-2214456ba24c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-bd17f6ca-d8ab-4657-aaa1-2214456ba24c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:43.323780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:43.324776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:43.325623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:43.325623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:43.325623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:43.325623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:43.325623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 259/510] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:43.325895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17f6ca-d8ab-4657-aaa1-2214456ba24c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.338396 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:43.350296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 252/511] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:43.351142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc7e58-5544-4725-b725-2cda29a8ccc3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.409040 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:43.427949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17f6ca-d8ab-4657-aaa1-2214456ba24c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.451014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc7e58-5544-4725-b725-2cda29a8ccc3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.461741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d97b72-cc76-4318-b5bd-cfead6c7953b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.462396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d97b72-cc76-4318-b5bd-cfead6c7953b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.462679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d97b72-cc76-4318-b5bd-cfead6c7953b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.462953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d97b72-cc76-4318-b5bd-cfead6c7953b None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.463705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d97b72-cc76-4318-b5bd-cfead6c7953b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:43.464008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d97b72-cc76-4318-b5bd-cfead6c7953b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.465395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d97b72-cc76-4318-b5bd-cfead6c7953b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.476166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 260/512] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:43.476939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d97b72-cc76-4318-b5bd-cfead6c7953b None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.489847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508c5b9-155e-4dc7-94e9-334e6af3f0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.490510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508c5b9-155e-4dc7-94e9-334e6af3f0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.490798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508c5b9-155e-4dc7-94e9-334e6af3f0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.491104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508c5b9-155e-4dc7-94e9-334e6af3f0b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.491843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508c5b9-155e-4dc7-94e9-334e6af3f0b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:43.492156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508c5b9-155e-4dc7-94e9-334e6af3f0b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.493516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508c5b9-155e-4dc7-94e9-334e6af3f0b2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.505082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 253/513] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:43.505784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508c5b9-155e-4dc7-94e9-334e6af3f0b2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.553894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:43.576916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d97b72-cc76-4318-b5bd-cfead6c7953b None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.603380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508c5b9-155e-4dc7-94e9-334e6af3f0b2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.610677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b9bb4-75ca-41f4-8687-93b692d0133d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.611484 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b9bb4-75ca-41f4-8687-93b692d0133d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.611773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b9bb4-75ca-41f4-8687-93b692d0133d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.612052 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b9bb4-75ca-41f4-8687-93b692d0133d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.622384 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b9bb4-75ca-41f4-8687-93b692d0133d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.623865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250b9bb4-75ca-41f4-8687-93b692d0133d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.625543 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:43.635831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65dc2b-1d25-48ef-bcf4-ca2c7c375fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.636483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65dc2b-1d25-48ef-bcf4-ca2c7c375fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.636763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65dc2b-1d25-48ef-bcf4-ca2c7c375fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.637738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65dc2b-1d25-48ef-bcf4-ca2c7c375fa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.638981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65dc2b-1d25-48ef-bcf4-ca2c7c375fa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:43.639575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65dc2b-1d25-48ef-bcf4-ca2c7c375fa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-250b9bb4-75ca-41f4-8687-93b692d0133d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:43.641837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:43.642647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:43.643562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:43.643562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:43.643562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:43.643562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:43.643773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65dc2b-1d25-48ef-bcf4-ca2c7c375fa5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.643840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 261/514] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:43.643840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b9bb4-75ca-41f4-8687-93b692d0133d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.655161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 254/515] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:43.655959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65dc2b-1d25-48ef-bcf4-ca2c7c375fa5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.748835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b9bb4-75ca-41f4-8687-93b692d0133d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.756485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65dc2b-1d25-48ef-bcf4-ca2c7c375fa5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.767632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:43.782165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802a31b-422f-4674-9852-b9998f5fc49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.782961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802a31b-422f-4674-9852-b9998f5fc49e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.783279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802a31b-422f-4674-9852-b9998f5fc49e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.783563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f802a31b-422f-4674-9852-b9998f5fc49e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.790818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caa785-079b-4223-a8e5-8116c0f18ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.791637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caa785-079b-4223-a8e5-8116c0f18ecc None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.791920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caa785-079b-4223-a8e5-8116c0f18ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.792204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46caa785-079b-4223-a8e5-8116c0f18ecc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.795342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802a31b-422f-4674-9852-b9998f5fc49e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.796822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f802a31b-422f-4674-9852-b9998f5fc49e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.799413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46caa785-079b-4223-a8e5-8116c0f18ecc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.800844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46caa785-079b-4223-a8e5-8116c0f18ecc None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-f802a31b-422f-4674-9852-b9998f5fc49e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:43.814194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:43.815068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:43.815916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:43.815916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:43.815916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:43.815916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:43.815916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 262/516] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-46caa785-079b-4223-a8e5-8116c0f18ecc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:43.816186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:43.817624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:43.818742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:43.818742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:43.818742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:43.818742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:43.818742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:43.818742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:43.818742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:43.818742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:43.819200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802a31b-422f-4674-9852-b9998f5fc49e None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.819281 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 255/517] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:43.819281 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caa785-079b-4223-a8e5-8116c0f18ecc None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.864701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:43.917167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46caa785-079b-4223-a8e5-8116c0f18ecc None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.917671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f802a31b-422f-4674-9852-b9998f5fc49e None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:43.949295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e2830-8d5c-4142-990b-8b4b9c00a277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.950118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa27163-52f8-46f5-a74e-e5ff5445a7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:43.950510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e2830-8d5c-4142-990b-8b4b9c00a277 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.950796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e2830-8d5c-4142-990b-8b4b9c00a277 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.950903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa27163-52f8-46f5-a74e-e5ff5445a7bf None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:43.951102 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e2830-8d5c-4142-990b-8b4b9c00a277 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.951472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa27163-52f8-46f5-a74e-e5ff5445a7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:43.951752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa27163-52f8-46f5-a74e-e5ff5445a7bf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:43.952079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e2830-8d5c-4142-990b-8b4b9c00a277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:43.952398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e2830-8d5c-4142-990b-8b4b9c00a277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.953771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e2830-8d5c-4142-990b-8b4b9c00a277 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.959252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa27163-52f8-46f5-a74e-e5ff5445a7bf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:43.960681 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa27163-52f8-46f5-a74e-e5ff5445a7bf None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:43.969471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 256/518] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:43.969471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e2830-8d5c-4142-990b-8b4b9c00a277 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-5aa27163-52f8-46f5-a74e-e5ff5445a7bf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:43.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:43.975665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:43.976571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 263/519] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:43.977135 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa27163-52f8-46f5-a74e-e5ff5445a7bf None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:44.000673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:44.069441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e2830-8d5c-4142-990b-8b4b9c00a277 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:44.079057 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa27163-52f8-46f5-a74e-e5ff5445a7bf None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:44.085720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:44.102477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d810b-66f5-4eea-b0d4-dd4652b76fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:44.103289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d810b-66f5-4eea-b0d4-dd4652b76fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:44.103571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d810b-66f5-4eea-b0d4-dd4652b76fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:44.103846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2d810b-66f5-4eea-b0d4-dd4652b76fd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:44.110269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d810b-66f5-4eea-b0d4-dd4652b76fd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:44.111736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2d810b-66f5-4eea-b0d4-dd4652b76fd2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:44.112277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c94eb-1e68-4562-b5be-91c7f8abfafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:44.112932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c94eb-1e68-4562-b5be-91c7f8abfafb None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:44.113223 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c94eb-1e68-4562-b5be-91c7f8abfafb None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:44.113509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c94eb-1e68-4562-b5be-91c7f8abfafb None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:44.114236 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c94eb-1e68-4562-b5be-91c7f8abfafb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:44.114546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c94eb-1e68-4562-b5be-91c7f8abfafb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:44.115929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c94eb-1e68-4562-b5be-91c7f8abfafb None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-7f2d810b-66f5-4eea-b0d4-dd4652b76fd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:44.125523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:44.126346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:44.127257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:44.127257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:44.127257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:44.127257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:44.127257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:44.127257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:44.127257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:44.127257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:44.127257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 257/520] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:44.127719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d810b-66f5-4eea-b0d4-dd4652b76fd2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:44.131342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 264/521] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:44.133241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c94eb-1e68-4562-b5be-91c7f8abfafb None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:44.216775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:44.227919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2d810b-66f5-4eea-b0d4-dd4652b76fd2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:44.233190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c94eb-1e68-4562-b5be-91c7f8abfafb None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:44.259990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876e463-2a27-478d-a1f7-b0cfbb776917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:44.260598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876e463-2a27-478d-a1f7-b0cfbb776917 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:44.260876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876e463-2a27-478d-a1f7-b0cfbb776917 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:44.261154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876e463-2a27-478d-a1f7-b0cfbb776917 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:44.262457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876e463-2a27-478d-a1f7-b0cfbb776917 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:44.262771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876e463-2a27-478d-a1f7-b0cfbb776917 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:44.264248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876e463-2a27-478d-a1f7-b0cfbb776917 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:44.266517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e97f0-90cd-4685-acb0-d13386ed9daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:44.267336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e97f0-90cd-4685-acb0-d13386ed9daa None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:44.267620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e97f0-90cd-4685-acb0-d13386ed9daa None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:44.267895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e97f0-90cd-4685-acb0-d13386ed9daa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:44.274703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e97f0-90cd-4685-acb0-d13386ed9daa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:44.276171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e97f0-90cd-4685-acb0-d13386ed9daa None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:44.281724 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 258/522] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:44.282352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876e463-2a27-478d-a1f7-b0cfbb776917 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-d54e97f0-90cd-4685-acb0-d13386ed9daa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:44.290340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:44.291388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 265/523] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:44.292290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e97f0-90cd-4685-acb0-d13386ed9daa None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:44.318818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:44.386499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876e463-2a27-478d-a1f7-b0cfbb776917 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:44.391579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e97f0-90cd-4685-acb0-d13386ed9daa None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:44.419572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc0f13-582f-4048-b417-70b27a166f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:44.420257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc0f13-582f-4048-b417-70b27a166f98 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:44.420543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc0f13-582f-4048-b417-70b27a166f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:44.420821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc0f13-582f-4048-b417-70b27a166f98 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:44.421655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc0f13-582f-4048-b417-70b27a166f98 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:44.421965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc0f13-582f-4048-b417-70b27a166f98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:44.423430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcc0f13-582f-4048-b417-70b27a166f98 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:44.425994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fa1d1-eb05-40a9-8940-e7c67497fe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:44.426660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa1d1-eb05-40a9-8940-e7c67497fe67 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:44.426946 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa1d1-eb05-40a9-8940-e7c67497fe67 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:44.427254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fa1d1-eb05-40a9-8940-e7c67497fe67 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:44.427967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fa1d1-eb05-40a9-8940-e7c67497fe67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:44.429391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fa1d1-eb05-40a9-8940-e7c67497fe67 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:44.441194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 259/524] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 5 15:00:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:44.442119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc0f13-582f-4048-b417-70b27a166f98 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:44.449864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 266/525] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:00:44 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:00:44.450683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fa1d1-eb05-40a9-8940-e7c67497fe67 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:44.459124 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:44.549810 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:44.558340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc0f13-582f-4048-b417-70b27a166f98 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:44.570079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fa1d1-eb05-40a9-8940-e7c67497fe67 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:44.602920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a661e5-357a-4995-8a40-4d4a53907462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:44.603616 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a661e5-357a-4995-8a40-4d4a53907462 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:44.603902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a661e5-357a-4995-8a40-4d4a53907462 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:44.604185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a661e5-357a-4995-8a40-4d4a53907462 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:44.605096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a661e5-357a-4995-8a40-4d4a53907462 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:44.605290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a661e5-357a-4995-8a40-4d4a53907462 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:44.607284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a661e5-357a-4995-8a40-4d4a53907462 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:44.608714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a30b9-8e0c-4641-bbf5-16797f4db608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:44.609540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a30b9-8e0c-4641-bbf5-16797f4db608 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:44.609830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a30b9-8e0c-4641-bbf5-16797f4db608 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:44.610110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a30b9-8e0c-4641-bbf5-16797f4db608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:44.611374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a30b9-8e0c-4641-bbf5-16797f4db608 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=e7a416f22d33450183801cef2b13822f) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:44.611374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a30b9-8e0c-4641-bbf5-16797f4db608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:44.612707 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a30b9-8e0c-4641-bbf5-16797f4db608 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:44.618600 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 267/526] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:44 2022] GET /identity/v3/role_assignments?user.id=e7a416f22d33450183801cef2b13822f&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:44.619246 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a30b9-8e0c-4641-bbf5-16797f4db608 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:44.624526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 260/527] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:44.624692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a661e5-357a-4995-8a40-4d4a53907462 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:44.717779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:44.766783 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a30b9-8e0c-4641-bbf5-16797f4db608 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:44.770928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a661e5-357a-4995-8a40-4d4a53907462 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:44.806115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95feb6-70d2-4cb2-b978-e7d7f841f202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:44.807028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95feb6-70d2-4cb2-b978-e7d7f841f202 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:44.807562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95feb6-70d2-4cb2-b978-e7d7f841f202 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:44.807942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c95feb6-70d2-4cb2-b978-e7d7f841f202 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:44.809232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95feb6-70d2-4cb2-b978-e7d7f841f202 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:44.809669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95feb6-70d2-4cb2-b978-e7d7f841f202 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:44.812174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c95feb6-70d2-4cb2-b978-e7d7f841f202 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:44.815197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c89efc-47a3-409d-a7ed-4e9ff2d9bb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:44.816040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c89efc-47a3-409d-a7ed-4e9ff2d9bb84 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:44.816470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c89efc-47a3-409d-a7ed-4e9ff2d9bb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:44.816889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c89efc-47a3-409d-a7ed-4e9ff2d9bb84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:44.861481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 268/528] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Jul 5 15:00:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:44.863108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95feb6-70d2-4cb2-b978-e7d7f841f202 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:44.875021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:44.903343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c89efc-47a3-409d-a7ed-4e9ff2d9bb84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:44.905505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c89efc-47a3-409d-a7ed-4e9ff2d9bb84 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:44.989361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:44.999086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c95feb6-70d2-4cb2-b978-e7d7f841f202 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:45.034825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 261/529] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:45.035756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c89efc-47a3-409d-a7ed-4e9ff2d9bb84 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:45.040313 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa010e6-34b7-4492-b6d1-0f9b17ad8406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:45.041205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa010e6-34b7-4492-b6d1-0f9b17ad8406 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:45.041693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa010e6-34b7-4492-b6d1-0f9b17ad8406 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:45.042187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa010e6-34b7-4492-b6d1-0f9b17ad8406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:45.043917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa010e6-34b7-4492-b6d1-0f9b17ad8406 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=450bcd0f22b840c299d08ce9433d6c85) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:45.044964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa010e6-34b7-4492-b6d1-0f9b17ad8406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:45.047168 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa010e6-34b7-4492-b6d1-0f9b17ad8406 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:45.055662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 269/530] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:44 2022] GET /identity/v3/role_assignments?user.id=450bcd0f22b840c299d08ce9433d6c85&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:45.056797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa010e6-34b7-4492-b6d1-0f9b17ad8406 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:45.140307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:45.185711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa010e6-34b7-4492-b6d1-0f9b17ad8406 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:45.227290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14d3f2-321d-4da9-9e36-78444a89652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:45.228103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14d3f2-321d-4da9-9e36-78444a89652e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:45.228391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14d3f2-321d-4da9-9e36-78444a89652e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:45.228669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14d3f2-321d-4da9-9e36-78444a89652e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:45.242409 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14d3f2-321d-4da9-9e36-78444a89652e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:45.244832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e14d3f2-321d-4da9-9e36-78444a89652e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-5e14d3f2-321d-4da9-9e36-78444a89652e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:45.266917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:45.267884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:45.268732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:45.268732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:45.268732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:45.268732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:45.269491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 270/531] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:45.274973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14d3f2-321d-4da9-9e36-78444a89652e None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:45.289373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c89efc-47a3-409d-a7ed-4e9ff2d9bb84 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:45.342102 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95a2e3-db49-43f5-81fa-765a8c443462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:45.344959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95a2e3-db49-43f5-81fa-765a8c443462 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:45.344959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95a2e3-db49-43f5-81fa-765a8c443462 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:45.344959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95a2e3-db49-43f5-81fa-765a8c443462 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:45.403444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:45.436002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95a2e3-db49-43f5-81fa-765a8c443462 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:45.437472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95a2e3-db49-43f5-81fa-765a8c443462 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:45.469804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e14d3f2-321d-4da9-9e36-78444a89652e None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:45.510156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:45.538494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407aa703-c5b0-4b2f-b66c-44529efe6158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:45.539188 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407aa703-c5b0-4b2f-b66c-44529efe6158 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:45.539478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407aa703-c5b0-4b2f-b66c-44529efe6158 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:45.539754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407aa703-c5b0-4b2f-b66c-44529efe6158 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:45.540487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407aa703-c5b0-4b2f-b66c-44529efe6158 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:45.540797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407aa703-c5b0-4b2f-b66c-44529efe6158 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:45.542181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407aa703-c5b0-4b2f-b66c-44529efe6158 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:45.565075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 271/532] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:45.571544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407aa703-c5b0-4b2f-b66c-44529efe6158 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:45.611056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 262/533] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:45.616559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95a2e3-db49-43f5-81fa-765a8c443462 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:45.803138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:45.853289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95a2e3-db49-43f5-81fa-765a8c443462 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:45.891507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407aa703-c5b0-4b2f-b66c-44529efe6158 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:45.900148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7b760-a391-4a1c-bf48-bef73525df0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:45.901945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7b760-a391-4a1c-bf48-bef73525df0a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:45.902235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7b760-a391-4a1c-bf48-bef73525df0a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:45.902617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7b760-a391-4a1c-bf48-bef73525df0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:45.918267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7b760-a391-4a1c-bf48-bef73525df0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:45.920407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7b760-a391-4a1c-bf48-bef73525df0a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-66b7b760-a391-4a1c-bf48-bef73525df0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:45.943176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:45.944271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:45.945411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:45.945411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:45.945411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:45.945411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:45.945411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 263/534] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:45.950744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7b760-a391-4a1c-bf48-bef73525df0a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:45.986167 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e835f3c-d27d-4979-bf25-ff0cd4b697f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:45.986974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e835f3c-d27d-4979-bf25-ff0cd4b697f5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:45.987284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e835f3c-d27d-4979-bf25-ff0cd4b697f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:45.987563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e835f3c-d27d-4979-bf25-ff0cd4b697f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:46.010398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e835f3c-d27d-4979-bf25-ff0cd4b697f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:46.017365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e835f3c-d27d-4979-bf25-ff0cd4b697f5 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-9e835f3c-d27d-4979-bf25-ff0cd4b697f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:46.058539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:46.059574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:46.060419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:46.060419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:46.060419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:46.060419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:46.060419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:46.060419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:46.060419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:46.060419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:46.064431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 272/535] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:46.067064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e835f3c-d27d-4979-bf25-ff0cd4b697f5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:46.217778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:46.279299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7b760-a391-4a1c-bf48-bef73525df0a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:46.315450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:46.346268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c623526-08f1-4155-af14-d7c2f478f053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:46.346942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c623526-08f1-4155-af14-d7c2f478f053 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:46.347254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c623526-08f1-4155-af14-d7c2f478f053 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:46.347541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c623526-08f1-4155-af14-d7c2f478f053 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:46.348280 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c623526-08f1-4155-af14-d7c2f478f053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:46.348590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c623526-08f1-4155-af14-d7c2f478f053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:46.349969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c623526-08f1-4155-af14-d7c2f478f053 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:46.363200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 264/536] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:46.369080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c623526-08f1-4155-af14-d7c2f478f053 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:46.400709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e835f3c-d27d-4979-bf25-ff0cd4b697f5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:46.476765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c623526-08f1-4155-af14-d7c2f478f053 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:46.489072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aa60b-19f9-4a9e-994e-0bc3a5001178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:46.489749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aa60b-19f9-4a9e-994e-0bc3a5001178 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:46.490038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aa60b-19f9-4a9e-994e-0bc3a5001178 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:46.490321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aa60b-19f9-4a9e-994e-0bc3a5001178 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:46.491154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aa60b-19f9-4a9e-994e-0bc3a5001178 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:46.491472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aa60b-19f9-4a9e-994e-0bc3a5001178 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:46.492930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aa60b-19f9-4a9e-994e-0bc3a5001178 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:46.512422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b50f7c-865f-4cc0-8493-3cd58fa2f91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:46.513229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b50f7c-865f-4cc0-8493-3cd58fa2f91d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:46.513518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b50f7c-865f-4cc0-8493-3cd58fa2f91d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:46.513929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b50f7c-865f-4cc0-8493-3cd58fa2f91d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:46.530178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b50f7c-865f-4cc0-8493-3cd58fa2f91d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:46.532685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b50f7c-865f-4cc0-8493-3cd58fa2f91d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:46.535631 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 273/537] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:46.541031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aa60b-19f9-4a9e-994e-0bc3a5001178 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-51b50f7c-865f-4cc0-8493-3cd58fa2f91d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:46.553590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:46.554457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:46.555437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:46.555437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:46.555437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:46.555437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:46.555437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:46.555437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:46.555437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:46.555437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:46.563653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 265/538] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:46.568006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b50f7c-865f-4cc0-8493-3cd58fa2f91d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:46.643427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:46.698479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aa60b-19f9-4a9e-994e-0bc3a5001178 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:46.703389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:46.760170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b50f7c-865f-4cc0-8493-3cd58fa2f91d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:46.767462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c7133-c412-40b8-9b49-3431f50291b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:46.768178 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c7133-c412-40b8-9b49-3431f50291b4 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:46.768468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c7133-c412-40b8-9b49-3431f50291b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:46.768746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c7133-c412-40b8-9b49-3431f50291b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:46.769590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c7133-c412-40b8-9b49-3431f50291b4 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=8ad7cee9ffed415b9a30176b1888e562) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:46.769906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c7133-c412-40b8-9b49-3431f50291b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:46.771390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c7133-c412-40b8-9b49-3431f50291b4 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:46.779066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 274/539] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:46 2022] GET /identity/v3/role_assignments?user.id=8ad7cee9ffed415b9a30176b1888e562&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:46.806197 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d3d89-b5bf-4f40-8f12-f69a9533132a None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:46.806780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d3d89-b5bf-4f40-8f12-f69a9533132a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:46.807259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d3d89-b5bf-4f40-8f12-f69a9533132a None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:46.809232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 275/540] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:46.815267 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48214a07-1248-45dc-961f-de59b2c8a276 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:46.815565 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48214a07-1248-45dc-961f-de59b2c8a276 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:46.815836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48214a07-1248-45dc-961f-de59b2c8a276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:46.822261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee026c-f289-43ea-90b5-02ccd8d5a93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:46.822956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee026c-f289-43ea-90b5-02ccd8d5a93f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:46.823330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee026c-f289-43ea-90b5-02ccd8d5a93f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:46.823608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee026c-f289-43ea-90b5-02ccd8d5a93f None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:46.825017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee026c-f289-43ea-90b5-02ccd8d5a93f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:46.825337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee026c-f289-43ea-90b5-02ccd8d5a93f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:46.827128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee026c-f289-43ea-90b5-02ccd8d5a93f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:46.850650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-48214a07-1248-45dc-961f-de59b2c8a276 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:46.851923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-48214a07-1248-45dc-961f-de59b2c8a276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:46.857796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 266/541] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:46.863687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee026c-f289-43ea-90b5-02ccd8d5a93f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:47.001719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 276/542] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:47.022146 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8ac24-5786-4a61-b1ae-35fbfe1e0cdf None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.022434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8ac24-5786-4a61-b1ae-35fbfe1e0cdf None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.022645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8ac24-5786-4a61-b1ae-35fbfe1e0cdf None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.024644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 277/543] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:00:47.030097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6044b-fd5d-4d33-8f0f-975f50415afe None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.030748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6044b-fd5d-4d33-8f0f-975f50415afe None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.031074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6044b-fd5d-4d33-8f0f-975f50415afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.033588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:47.040997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee026c-f289-43ea-90b5-02ccd8d5a93f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:47.052780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5ab6044b-fd5d-4d33-8f0f-975f50415afe None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:47.053971 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5ab6044b-fd5d-4d33-8f0f-975f50415afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:47.077521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd274a3c-d287-430d-9769-60b3c0991a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:47.078536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd274a3c-d287-430d-9769-60b3c0991a93 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.079034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd274a3c-d287-430d-9769-60b3c0991a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.079540 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd274a3c-d287-430d-9769-60b3c0991a93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.080804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd274a3c-d287-430d-9769-60b3c0991a93 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=cd7fa93466c54632ad95834297410e00) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:47.081324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd274a3c-d287-430d-9769-60b3c0991a93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:47.083273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd274a3c-d287-430d-9769-60b3c0991a93 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:47.089719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 267/544] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:46 2022] GET /identity/v3/role_assignments?user.id=cd7fa93466c54632ad95834297410e00&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:47.100480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd274a3c-d287-430d-9769-60b3c0991a93 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:47.129583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 278/545] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:47.156061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6044b-fd5d-4d33-8f0f-975f50415afe None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:47.183585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:47.267173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6044b-fd5d-4d33-8f0f-975f50415afe None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:47.304030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af6cf-f3e6-4bb2-986d-4022bab57bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:47.304768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af6cf-f3e6-4bb2-986d-4022bab57bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.305051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af6cf-f3e6-4bb2-986d-4022bab57bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.305326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767af6cf-f3e6-4bb2-986d-4022bab57bc5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.317315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767af6cf-f3e6-4bb2-986d-4022bab57bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-767af6cf-f3e6-4bb2-986d-4022bab57bc5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:47.325908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:47.327160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:47.327959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 279/546] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:47.336595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af6cf-f3e6-4bb2-986d-4022bab57bc5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:47.350596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd274a3c-d287-430d-9769-60b3c0991a93 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:47.399452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79207e46-cbbd-4240-982c-06af6c1ec853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:47.400314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79207e46-cbbd-4240-982c-06af6c1ec853 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.400607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79207e46-cbbd-4240-982c-06af6c1ec853 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.400885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79207e46-cbbd-4240-982c-06af6c1ec853 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.424963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79207e46-cbbd-4240-982c-06af6c1ec853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-79207e46-cbbd-4240-982c-06af6c1ec853 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:47.432750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:47.433719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:47.434522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:47.451481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 268/547] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:47.452425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79207e46-cbbd-4240-982c-06af6c1ec853 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:47.480563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:47.500972 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767af6cf-f3e6-4bb2-986d-4022bab57bc5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:47.502444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:47.549012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5c769-0456-4562-b15c-fd2198906668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:47.550440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5c769-0456-4562-b15c-fd2198906668 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.551342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5c769-0456-4562-b15c-fd2198906668 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.551669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5c769-0456-4562-b15c-fd2198906668 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.552427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5c769-0456-4562-b15c-fd2198906668 None admin] RBAC: Adding query filter params (name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:47.552740 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5c769-0456-4562-b15c-fd2198906668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:47.554193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5c769-0456-4562-b15c-fd2198906668 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:47.567067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 280/548] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:47.567067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474487c-fe3e-454b-8387-8b52e1e99fa4 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.567067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474487c-fe3e-454b-8387-8b52e1e99fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.567067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474487c-fe3e-454b-8387-8b52e1e99fa4 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.567067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 281/549] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:47.574136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474487c-fe3e-454b-8387-8b52e1e99fa4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:47.607000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79207e46-cbbd-4240-982c-06af6c1ec853 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:47.650295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb0844-d3f4-48f5-b27f-23227ff60110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:47.651319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb0844-d3f4-48f5-b27f-23227ff60110 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.651672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb0844-d3f4-48f5-b27f-23227ff60110 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.651956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb0844-d3f4-48f5-b27f-23227ff60110 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.653319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb0844-d3f4-48f5-b27f-23227ff60110 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:47.653881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb0844-d3f4-48f5-b27f-23227ff60110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:47.655875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deb0844-d3f4-48f5-b27f-23227ff60110 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:47.667120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 269/550] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:47.669234 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb22dd-970a-443a-bd78-353ba3044657 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.670182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb22dd-970a-443a-bd78-353ba3044657 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.670956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb22dd-970a-443a-bd78-353ba3044657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.694356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-19fb22dd-970a-443a-bd78-353ba3044657 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:47.696125 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-19fb22dd-970a-443a-bd78-353ba3044657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:47.712245 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474487c-fe3e-454b-8387-8b52e1e99fa4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:47.758673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77c58f-ca0a-456d-86c1-91b3ea0bfe1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:47.760211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77c58f-ca0a-456d-86c1-91b3ea0bfe1f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.761065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77c58f-ca0a-456d-86c1-91b3ea0bfe1f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.761603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77c58f-ca0a-456d-86c1-91b3ea0bfe1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.776940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77c58f-ca0a-456d-86c1-91b3ea0bfe1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:47.779904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee77c58f-ca0a-456d-86c1-91b3ea0bfe1f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:47.783165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 270/551] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:47.784303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb22dd-970a-443a-bd78-353ba3044657 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:47.798477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:47.815908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-ee77c58f-ca0a-456d-86c1-91b3ea0bfe1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:47.818346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:47.819713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:47.822034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:47.822034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:47.822034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:47.822034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:47.825768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 282/552] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:47.830888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77c58f-ca0a-456d-86c1-91b3ea0bfe1f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:47.915213 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb22dd-970a-443a-bd78-353ba3044657 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:47.936842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee77c58f-ca0a-456d-86c1-91b3ea0bfe1f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:47.956977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a0731-9391-42b0-aea6-a4cf4daeee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:47.958119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a0731-9391-42b0-aea6-a4cf4daeee0e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.958609 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a0731-9391-42b0-aea6-a4cf4daeee0e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.959107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a0731-9391-42b0-aea6-a4cf4daeee0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.971031 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a0731-9391-42b0-aea6-a4cf4daeee0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:47.973012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337801e-dd92-47e8-90ab-278707ad95cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:47.973420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2a0731-9391-42b0-aea6-a4cf4daeee0e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:47.973818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337801e-dd92-47e8-90ab-278707ad95cc None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:47.974103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337801e-dd92-47e8-90ab-278707ad95cc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:47.974405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337801e-dd92-47e8-90ab-278707ad95cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:47.985769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e337801e-dd92-47e8-90ab-278707ad95cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-2e2a0731-9391-42b0-aea6-a4cf4daeee0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:47.993221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:47.994067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:47.995127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:47.995127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:47.995127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:47.995127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-e337801e-dd92-47e8-90ab-278707ad95cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:47.995341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:47.996259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:47.998121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:47.999601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 283/553] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:48.000245 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337801e-dd92-47e8-90ab-278707ad95cc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.001630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 271/554] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:48.002607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a0731-9391-42b0-aea6-a4cf4daeee0e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.065393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:48.072415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:48.110611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337801e-dd92-47e8-90ab-278707ad95cc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:48.111201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a0731-9391-42b0-aea6-a4cf4daeee0e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:48.145615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a733c-ee23-46af-89d3-0e0f480a86e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:48.146248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00558c-5ed6-4514-ad10-db0f59ece3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:48.146459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a733c-ee23-46af-89d3-0e0f480a86e8 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.146541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a733c-ee23-46af-89d3-0e0f480a86e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.146828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a733c-ee23-46af-89d3-0e0f480a86e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.147591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a733c-ee23-46af-89d3-0e0f480a86e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:48.147836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00558c-5ed6-4514-ad10-db0f59ece3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.147950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a733c-ee23-46af-89d3-0e0f480a86e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:48.148409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00558c-5ed6-4514-ad10-db0f59ece3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.148874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00558c-5ed6-4514-ad10-db0f59ece3d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.149273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a733c-ee23-46af-89d3-0e0f480a86e8 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:48.150018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00558c-5ed6-4514-ad10-db0f59ece3d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:48.150530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00558c-5ed6-4514-ad10-db0f59ece3d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:48.152106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc00558c-5ed6-4514-ad10-db0f59ece3d7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:48.160699 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 272/555] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:48.161706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00558c-5ed6-4514-ad10-db0f59ece3d7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.166876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 284/556] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:48.167729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a733c-ee23-46af-89d3-0e0f480a86e8 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.267935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00558c-5ed6-4514-ad10-db0f59ece3d7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:48.277560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a733c-ee23-46af-89d3-0e0f480a86e8 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:48.296121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:48.298800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:48.306323 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe21f1-2b3f-4dda-80bc-debcda03f831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:48.307412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe21f1-2b3f-4dda-80bc-debcda03f831 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.307844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe21f1-2b3f-4dda-80bc-debcda03f831 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.308289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe21f1-2b3f-4dda-80bc-debcda03f831 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.310098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe21f1-2b3f-4dda-80bc-debcda03f831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:48.310740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe21f1-2b3f-4dda-80bc-debcda03f831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:48.312164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe21f1-2b3f-4dda-80bc-debcda03f831 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:48.315676 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e446-3e97-4494-8619-a8f449173f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:48.316689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e446-3e97-4494-8619-a8f449173f9a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.317155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e446-3e97-4494-8619-a8f449173f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.317640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370e446-3e97-4494-8619-a8f449173f9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.332468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 273/557] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:48.332468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe21f1-2b3f-4dda-80bc-debcda03f831 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.335222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370e446-3e97-4494-8619-a8f449173f9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:48.336991 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370e446-3e97-4494-8619-a8f449173f9a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-2370e446-3e97-4494-8619-a8f449173f9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:48.356261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:48.357743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:48.358629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:48.358629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:48.358629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:48.358629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:48.365196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 285/558] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:48.366288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e446-3e97-4494-8619-a8f449173f9a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.438419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe21f1-2b3f-4dda-80bc-debcda03f831 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:48.468848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370e446-3e97-4494-8619-a8f449173f9a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:48.475691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0abde1-ad26-4078-be9a-1cdd791e31cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:48.476558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0abde1-ad26-4078-be9a-1cdd791e31cb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.476911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0abde1-ad26-4078-be9a-1cdd791e31cb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.477586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0abde1-ad26-4078-be9a-1cdd791e31cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.485442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0abde1-ad26-4078-be9a-1cdd791e31cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:48.486879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0abde1-ad26-4078-be9a-1cdd791e31cb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ef0abde1-ad26-4078-be9a-1cdd791e31cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:48.503160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:48.504131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:48.505155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:48.505155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:48.505155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:48.505155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:48.505155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:48.505155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:48.505155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:48.505155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:48.505155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:00:48.505155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:48.507113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 274/559] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:48.508129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0abde1-ad26-4078-be9a-1cdd791e31cb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.511507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921e8cf-e970-43f0-94c9-4b17b032b98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:48.512631 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921e8cf-e970-43f0-94c9-4b17b032b98e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.513141 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921e8cf-e970-43f0-94c9-4b17b032b98e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.513621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921e8cf-e970-43f0-94c9-4b17b032b98e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.521636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921e8cf-e970-43f0-94c9-4b17b032b98e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:48.523373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921e8cf-e970-43f0-94c9-4b17b032b98e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-e921e8cf-e970-43f0-94c9-4b17b032b98e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:48.538536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:48.539758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:48.540714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:48.540714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:48.540714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:48.540714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:48.540714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:48.540714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:48.540714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:48.540714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:48.540714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:00:48.540714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:48.547615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 286/560] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:48.548708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921e8cf-e970-43f0-94c9-4b17b032b98e None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.563484 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:48.580555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:48.621896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0abde1-ad26-4078-be9a-1cdd791e31cb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:48.653189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921e8cf-e970-43f0-94c9-4b17b032b98e None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:48.658269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f2da6-4648-47f7-9c2a-49569b1e14f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:48.658943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f2da6-4648-47f7-9c2a-49569b1e14f0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.659266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f2da6-4648-47f7-9c2a-49569b1e14f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.659545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7f2da6-4648-47f7-9c2a-49569b1e14f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.660272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f2da6-4648-47f7-9c2a-49569b1e14f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:48.660581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f2da6-4648-47f7-9c2a-49569b1e14f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:48.662348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7f2da6-4648-47f7-9c2a-49569b1e14f0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:48.679136 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 275/561] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:48.679136 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f2da6-4648-47f7-9c2a-49569b1e14f0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.695294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e248681-5667-4816-9954-1d3091a7b323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:48.696273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e248681-5667-4816-9954-1d3091a7b323 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.696764 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e248681-5667-4816-9954-1d3091a7b323 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.697271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e248681-5667-4816-9954-1d3091a7b323 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.698317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e248681-5667-4816-9954-1d3091a7b323 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:48.698834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e248681-5667-4816-9954-1d3091a7b323 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:48.700580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e248681-5667-4816-9954-1d3091a7b323 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:48.720184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 287/562] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:48.721352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e248681-5667-4816-9954-1d3091a7b323 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.792653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7f2da6-4648-47f7-9c2a-49569b1e14f0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:48.803889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:48.814770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:48.833265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39afe01-eefc-4cf3-b7bf-66a8af3af7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:48.834087 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39afe01-eefc-4cf3-b7bf-66a8af3af7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.834473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39afe01-eefc-4cf3-b7bf-66a8af3af7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.834870 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39afe01-eefc-4cf3-b7bf-66a8af3af7d4 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.835035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e248681-5667-4816-9954-1d3091a7b323 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:48.835952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39afe01-eefc-4cf3-b7bf-66a8af3af7d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:48.836367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39afe01-eefc-4cf3-b7bf-66a8af3af7d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:48.838536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39afe01-eefc-4cf3-b7bf-66a8af3af7d4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:48.856533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 276/563] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:48.857508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39afe01-eefc-4cf3-b7bf-66a8af3af7d4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.876761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb90b8-65c7-458e-a650-b63dde55b370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:48.877491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb90b8-65c7-458e-a650-b63dde55b370 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.877781 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb90b8-65c7-458e-a650-b63dde55b370 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.878060 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb90b8-65c7-458e-a650-b63dde55b370 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.885343 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb90b8-65c7-458e-a650-b63dde55b370 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:48.886825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb90b8-65c7-458e-a650-b63dde55b370 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-4afb90b8-65c7-458e-a650-b63dde55b370 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:48.902013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:48.903111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:48.904065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 288/564] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:48.905080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11502f5b-ac84-4028-8ba4-6214eb41fafd None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:48.905080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11502f5b-ac84-4028-8ba4-6214eb41fafd None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:48.905390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11502f5b-ac84-4028-8ba4-6214eb41fafd None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:48.906766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 289/565] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:48.911291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11502f5b-ac84-4028-8ba4-6214eb41fafd None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:48.975152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39afe01-eefc-4cf3-b7bf-66a8af3af7d4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:49.017451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11502f5b-ac84-4028-8ba4-6214eb41fafd None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:49.018413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1708c1-92ca-419c-bff3-f3e798b48152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:49.019249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1708c1-92ca-419c-bff3-f3e798b48152 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.019542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1708c1-92ca-419c-bff3-f3e798b48152 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.019830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1708c1-92ca-419c-bff3-f3e798b48152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.031347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1708c1-92ca-419c-bff3-f3e798b48152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:49.032828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1708c1-92ca-419c-bff3-f3e798b48152 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:49.054461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadec39f-7a10-48b3-982c-3d33d09b0669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:49.055445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadec39f-7a10-48b3-982c-3d33d09b0669 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.055915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadec39f-7a10-48b3-982c-3d33d09b0669 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-3b1708c1-92ca-419c-bff3-f3e798b48152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:49.056132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:49.057047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:49.058006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:49.058006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:49.058006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:49.058006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:49.058819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadec39f-7a10-48b3-982c-3d33d09b0669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.062849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 277/566] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:49.063728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1708c1-92ca-419c-bff3-f3e798b48152 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:49.079712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:49.094266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:49.142441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadec39f-7a10-48b3-982c-3d33d09b0669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:49.143947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadec39f-7a10-48b3-982c-3d33d09b0669 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:49.182842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1708c1-92ca-419c-bff3-f3e798b48152 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:49.223509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ef46ab-3d8c-4ff6-a5cf-6455c09c1aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:49.224184 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef46ab-3d8c-4ff6-a5cf-6455c09c1aab None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.224475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef46ab-3d8c-4ff6-a5cf-6455c09c1aab None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.224747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ef46ab-3d8c-4ff6-a5cf-6455c09c1aab None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.225600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef46ab-3d8c-4ff6-a5cf-6455c09c1aab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:49.225916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef46ab-3d8c-4ff6-a5cf-6455c09c1aab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:49.227373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ef46ab-3d8c-4ff6-a5cf-6455c09c1aab None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:49.249396 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 278/567] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 5 15:00:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:49.249396 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8050a-c024-4b92-bb52-a3b193cef8b3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.249396 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8050a-c024-4b92-bb52-a3b193cef8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.249396 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8050a-c024-4b92-bb52-a3b193cef8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.259374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 290/568] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:49.259374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadec39f-7a10-48b3-982c-3d33d09b0669 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:49.264269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-94a8050a-c024-4b92-bb52-a3b193cef8b3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:49.265472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-94a8050a-c024-4b92-bb52-a3b193cef8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:49.304448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:49.329538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 279/569] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:49.330901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e4ee8-fc66-4c1d-9b23-1847c1836549 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.331220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e4ee8-fc66-4c1d-9b23-1847c1836549 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.331497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2e4ee8-fc66-4c1d-9b23-1847c1836549 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.335156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 280/570] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:49.336107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e4ee8-fc66-4c1d-9b23-1847c1836549 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:49.354887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:49.355312 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadec39f-7a10-48b3-982c-3d33d09b0669 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:49.389788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218653ec-b451-4cbe-8200-e80065cac89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:49.390449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218653ec-b451-4cbe-8200-e80065cac89e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.390735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218653ec-b451-4cbe-8200-e80065cac89e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.391023 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218653ec-b451-4cbe-8200-e80065cac89e None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.391770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218653ec-b451-4cbe-8200-e80065cac89e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:49.392083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218653ec-b451-4cbe-8200-e80065cac89e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:49.394346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218653ec-b451-4cbe-8200-e80065cac89e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:49.410328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 291/571] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:49.411212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218653ec-b451-4cbe-8200-e80065cac89e None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:49.451378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2e4ee8-fc66-4c1d-9b23-1847c1836549 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:49.484842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c0c87-42b4-4df9-be0f-7665e2b5d3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:49.485631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c0c87-42b4-4df9-be0f-7665e2b5d3fe None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.485914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c0c87-42b4-4df9-be0f-7665e2b5d3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.486186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c0c87-42b4-4df9-be0f-7665e2b5d3fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.495802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c0c87-42b4-4df9-be0f-7665e2b5d3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:49.497245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c0c87-42b4-4df9-be0f-7665e2b5d3fe None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:49.510057 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218653ec-b451-4cbe-8200-e80065cac89e None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ff4c0c87-42b4-4df9-be0f-7665e2b5d3fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:49.515097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:49.516165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:49.517366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:49.517366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:49.517366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:49.517366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:49.519370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 281/572] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:49.519477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:49.520480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53ebe2-b441-4a9d-9a1f-fd59bc5be73b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.520480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53ebe2-b441-4a9d-9a1f-fd59bc5be73b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.520982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53ebe2-b441-4a9d-9a1f-fd59bc5be73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.537221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-dd53ebe2-b441-4a9d-9a1f-fd59bc5be73b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:49.538409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-dd53ebe2-b441-4a9d-9a1f-fd59bc5be73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:49.545739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3984b-1bc5-4afc-9816-8854a77c31fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:49.546525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3984b-1bc5-4afc-9816-8854a77c31fc None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.546812 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3984b-1bc5-4afc-9816-8854a77c31fc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.547120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3984b-1bc5-4afc-9816-8854a77c31fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.557630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3984b-1bc5-4afc-9816-8854a77c31fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:49.559144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3984b-1bc5-4afc-9816-8854a77c31fc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-a4d3984b-1bc5-4afc-9816-8854a77c31fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:49.576430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:49.577475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:49.578371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:49.578371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:49.578371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:49.578371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:49.578371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 292/573] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:49.578371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3984b-1bc5-4afc-9816-8854a77c31fc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:49.594181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:49.606897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 282/574] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:49.607735 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53ebe2-b441-4a9d-9a1f-fd59bc5be73b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:49.679663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3984b-1bc5-4afc-9816-8854a77c31fc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:49.704294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53ebe2-b441-4a9d-9a1f-fd59bc5be73b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:49.712320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c2b53-8490-48be-a636-b179b0ec7527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:49.713114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c2b53-8490-48be-a636-b179b0ec7527 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.713398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c2b53-8490-48be-a636-b179b0ec7527 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.713675 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c2b53-8490-48be-a636-b179b0ec7527 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.724062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c2b53-8490-48be-a636-b179b0ec7527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-c32c2b53-8490-48be-a636-b179b0ec7527 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:49.731796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:49.732850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 293/575] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:49.733719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c2b53-8490-48be-a636-b179b0ec7527 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:49.742054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4677f-2a31-424a-b88c-1591ce7e0273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:49.742838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4677f-2a31-424a-b88c-1591ce7e0273 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.743147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4677f-2a31-424a-b88c-1591ce7e0273 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.743423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4677f-2a31-424a-b88c-1591ce7e0273 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.750276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4677f-2a31-424a-b88c-1591ce7e0273 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:49.751781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4677f-2a31-424a-b88c-1591ce7e0273 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:49.758131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f8b4677f-2a31-424a-b88c-1591ce7e0273 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:49.766155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:49.767134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:49.768053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:49.768053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:49.768053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:49.768053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:49.768053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:49.768053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:49.768053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:49.768053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:49.768053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 283/576] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:49.768554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4677f-2a31-424a-b88c-1591ce7e0273 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:49.817000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:49.842099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c2b53-8490-48be-a636-b179b0ec7527 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:49.866398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4677f-2a31-424a-b88c-1591ce7e0273 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:49.878053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ba188-a2b8-4517-be49-ca0900633893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:49.878715 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ba188-a2b8-4517-be49-ca0900633893 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.879000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ba188-a2b8-4517-be49-ca0900633893 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.879306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ba188-a2b8-4517-be49-ca0900633893 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.880038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ba188-a2b8-4517-be49-ca0900633893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:49.880345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ba188-a2b8-4517-be49-ca0900633893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:49.881730 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ba188-a2b8-4517-be49-ca0900633893 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:49.894331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 294/577] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:49.895152 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ba188-a2b8-4517-be49-ca0900633893 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:49.901923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bda25e-a717-48f9-8dc4-25d43410318b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:49.902625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bda25e-a717-48f9-8dc4-25d43410318b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:49.902910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bda25e-a717-48f9-8dc4-25d43410318b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:49.903218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bda25e-a717-48f9-8dc4-25d43410318b None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:49.903948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bda25e-a717-48f9-8dc4-25d43410318b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:49.904254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bda25e-a717-48f9-8dc4-25d43410318b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:49.905604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bda25e-a717-48f9-8dc4-25d43410318b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:49.917502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 284/578] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:49.918154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bda25e-a717-48f9-8dc4-25d43410318b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:49.977153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:49.995880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ba188-a2b8-4517-be49-ca0900633893 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.015207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bda25e-a717-48f9-8dc4-25d43410318b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.025395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:50.030398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba836f29-c34a-498a-944b-423d5715bb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.031224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba836f29-c34a-498a-944b-423d5715bb9d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.031517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba836f29-c34a-498a-944b-423d5715bb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.031794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba836f29-c34a-498a-944b-423d5715bb9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.041682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba836f29-c34a-498a-944b-423d5715bb9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:50.048230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940171d-5b73-4aab-bc1f-a17ddf5444f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.048892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940171d-5b73-4aab-bc1f-a17ddf5444f8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.049176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940171d-5b73-4aab-bc1f-a17ddf5444f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.049451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940171d-5b73-4aab-bc1f-a17ddf5444f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-ba836f29-c34a-498a-944b-423d5715bb9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:50.049809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:50.050882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:50.051804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:50.052437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940171d-5b73-4aab-bc1f-a17ddf5444f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:50.052437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940171d-5b73-4aab-bc1f-a17ddf5444f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.052568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 295/579] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:50.052642 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940171d-5b73-4aab-bc1f-a17ddf5444f8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.052719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba836f29-c34a-498a-944b-423d5715bb9d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.064605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 285/580] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:50.064605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940171d-5b73-4aab-bc1f-a17ddf5444f8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.151639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba836f29-c34a-498a-944b-423d5715bb9d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.157038 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940171d-5b73-4aab-bc1f-a17ddf5444f8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.184809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117d5cb-2fab-455b-9d6b-62502a191768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.185463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117d5cb-2fab-455b-9d6b-62502a191768 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.185779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117d5cb-2fab-455b-9d6b-62502a191768 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.186079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117d5cb-2fab-455b-9d6b-62502a191768 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.186939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117d5cb-2fab-455b-9d6b-62502a191768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:50.187288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117d5cb-2fab-455b-9d6b-62502a191768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.188890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117d5cb-2fab-455b-9d6b-62502a191768 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.189656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb9772-5e87-44c9-92ba-929b586b74c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.190617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb9772-5e87-44c9-92ba-929b586b74c0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.190913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb9772-5e87-44c9-92ba-929b586b74c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.191228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb9772-5e87-44c9-92ba-929b586b74c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.198181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:50.200094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb9772-5e87-44c9-92ba-929b586b74c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.201858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb9772-5e87-44c9-92ba-929b586b74c0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.202831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 296/581] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:50.203903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117d5cb-2fab-455b-9d6b-62502a191768 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c9fb9772-5e87-44c9-92ba-929b586b74c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:50.221783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:50.222771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:50.224006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:50.224006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:50.224006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:50.224006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:50.224006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:50.224006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:50.224006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:50.224006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:50.224006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 286/582] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:50.224006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb9772-5e87-44c9-92ba-929b586b74c0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.255255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:50.304316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117d5cb-2fab-455b-9d6b-62502a191768 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.321514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb9772-5e87-44c9-92ba-929b586b74c0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.337653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e3af-112f-4aeb-8e1b-a5b571f0f4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.338440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e3af-112f-4aeb-8e1b-a5b571f0f4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.338726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e3af-112f-4aeb-8e1b-a5b571f0f4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.339000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e3af-112f-4aeb-8e1b-a5b571f0f4a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.345623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105e3af-112f-4aeb-8e1b-a5b571f0f4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.347092 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105e3af-112f-4aeb-8e1b-a5b571f0f4a7 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.354771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46922b-4ce6-4f3a-bca4-9884f0ed514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.355970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46922b-4ce6-4f3a-bca4-9884f0ed514d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.356249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46922b-4ce6-4f3a-bca4-9884f0ed514d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.356522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46922b-4ce6-4f3a-bca4-9884f0ed514d None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.357256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46922b-4ce6-4f3a-bca4-9884f0ed514d None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:50.357558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46922b-4ce6-4f3a-bca4-9884f0ed514d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.358893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46922b-4ce6-4f3a-bca4-9884f0ed514d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-4105e3af-112f-4aeb-8e1b-a5b571f0f4a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:50.361319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:50.362335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:50.363303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:50.363303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:50.363303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:50.363303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:50.363303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:50.363303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:50.363303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:50.363303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:50.363303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 297/583] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:50.363303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e3af-112f-4aeb-8e1b-a5b571f0f4a7 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.370415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 287/584] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:50.371245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46922b-4ce6-4f3a-bca4-9884f0ed514d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.424329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:50.461840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:50.468140 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e3af-112f-4aeb-8e1b-a5b571f0f4a7 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.469146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46922b-4ce6-4f3a-bca4-9884f0ed514d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.501748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676cb7e-f686-4874-9d37-b5463fecd3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.502103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e52004-58d1-4aef-9e37-9dc80a26e3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.502544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676cb7e-f686-4874-9d37-b5463fecd3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.502831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676cb7e-f686-4874-9d37-b5463fecd3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.502982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e52004-58d1-4aef-9e37-9dc80a26e3a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.503138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676cb7e-f686-4874-9d37-b5463fecd3c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.503297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e52004-58d1-4aef-9e37-9dc80a26e3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.503576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e52004-58d1-4aef-9e37-9dc80a26e3a0 None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/e7a416f22d33450183801cef2b13822f/roles/67e4864efce9469bb6bdcca8ee5d5b24` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.513724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676cb7e-f686-4874-9d37-b5463fecd3c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.515175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9676cb7e-f686-4874-9d37-b5463fecd3c6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.517484 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e52004-58d1-4aef-9e37-9dc80a26e3a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=e7a416f22d33450183801cef2b13822f, role_id=67e4864efce9469bb6bdcca8ee5d5b24)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.518973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e52004-58d1-4aef-9e37-9dc80a26e3a0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.530979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 288/585] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:00:50 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/e7a416f22d33450183801cef2b13822f/roles/67e4864efce9469bb6bdcca8ee5d5b24 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:50.531795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e52004-58d1-4aef-9e37-9dc80a26e3a0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-9676cb7e-f686-4874-9d37-b5463fecd3c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:50.532967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:50.533851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:50.534824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:50.534824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:50.534824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:50.534824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:50.534824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 298/586] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:50.535155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676cb7e-f686-4874-9d37-b5463fecd3c6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.655997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e52004-58d1-4aef-9e37-9dc80a26e3a0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.664170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676cb7e-f686-4874-9d37-b5463fecd3c6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.680898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:50.703344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13c11a-7824-416f-9bfc-7effdb16dc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.704102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0420b-cf01-4a3a-8e2d-a71eeb0e5202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.704542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13c11a-7824-416f-9bfc-7effdb16dc54 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.704771 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0420b-cf01-4a3a-8e2d-a71eeb0e5202 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.705063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0420b-cf01-4a3a-8e2d-a71eeb0e5202 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.705346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0420b-cf01-4a3a-8e2d-a71eeb0e5202 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.705552 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13c11a-7824-416f-9bfc-7effdb16dc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.705992 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13c11a-7824-416f-9bfc-7effdb16dc54 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.706148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0420b-cf01-4a3a-8e2d-a71eeb0e5202 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:50.706460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0420b-cf01-4a3a-8e2d-a71eeb0e5202 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.707312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13c11a-7824-416f-9bfc-7effdb16dc54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:50.707798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13c11a-7824-416f-9bfc-7effdb16dc54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.707984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0420b-cf01-4a3a-8e2d-a71eeb0e5202 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.709556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13c11a-7824-416f-9bfc-7effdb16dc54 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.720708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 299/587] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:50.721825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0420b-cf01-4a3a-8e2d-a71eeb0e5202 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.722113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 289/588] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:50.723064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13c11a-7824-416f-9bfc-7effdb16dc54 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.729919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:50.833170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0420b-cf01-4a3a-8e2d-a71eeb0e5202 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.838731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13c11a-7824-416f-9bfc-7effdb16dc54 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:50.869420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c99475-0327-419d-a28f-8479d27a04b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.870294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c99475-0327-419d-a28f-8479d27a04b9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.870591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c99475-0327-419d-a28f-8479d27a04b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.870860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c99475-0327-419d-a28f-8479d27a04b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.875167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0082c9-9e41-412f-bb12-037a205599dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:50.875887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0082c9-9e41-412f-bb12-037a205599dc None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:50.876177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0082c9-9e41-412f-bb12-037a205599dc None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:50.876460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0082c9-9e41-412f-bb12-037a205599dc None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:50.877192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0082c9-9e41-412f-bb12-037a205599dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:50.877504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0082c9-9e41-412f-bb12-037a205599dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.878879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0082c9-9e41-412f-bb12-037a205599dc None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.882255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c99475-0327-419d-a28f-8479d27a04b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:50.883826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c99475-0327-419d-a28f-8479d27a04b9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:50.895302 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 290/589] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:50.895302 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0082c9-9e41-412f-bb12-037a205599dc None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-82c99475-0327-419d-a28f-8479d27a04b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:50.907909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:50.909238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:50.910202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:50.910202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:50.910202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:50.910202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:50.910202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 300/590] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:50.910202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c99475-0327-419d-a28f-8479d27a04b9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:50.937418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:50.966399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:51.008544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0082c9-9e41-412f-bb12-037a205599dc None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:51.021292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c99475-0327-419d-a28f-8479d27a04b9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:51.042513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5f1ca-2376-4241-aed5-d506e1c6e975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:51.044002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5f1ca-2376-4241-aed5-d506e1c6e975 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:51.044246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5f1ca-2376-4241-aed5-d506e1c6e975 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:51.044503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5f1ca-2376-4241-aed5-d506e1c6e975 None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/450bcd0f22b840c299d08ce9433d6c85/roles/67e4864efce9469bb6bdcca8ee5d5b24` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:51.058912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5f1ca-2376-4241-aed5-d506e1c6e975 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=450bcd0f22b840c299d08ce9433d6c85, role_id=67e4864efce9469bb6bdcca8ee5d5b24)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:51.059822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9217945-d1f7-4cda-b39f-892eca73a877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:51.060473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9217945-d1f7-4cda-b39f-892eca73a877 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:51.060573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5f1ca-2376-4241-aed5-d506e1c6e975 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:51.060764 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9217945-d1f7-4cda-b39f-892eca73a877 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:51.061038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9217945-d1f7-4cda-b39f-892eca73a877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:51.061870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9217945-d1f7-4cda-b39f-892eca73a877 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=4c73aa1f244d4251b8539c2806f38a42) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:51.062186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9217945-d1f7-4cda-b39f-892eca73a877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:51.063660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9217945-d1f7-4cda-b39f-892eca73a877 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:51.069819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 301/591] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:50 2022] GET /identity/v3/role_assignments?user.id=4c73aa1f244d4251b8539c2806f38a42&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:51.070486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9217945-d1f7-4cda-b39f-892eca73a877 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:51.083993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 291/592] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:00:50 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/450bcd0f22b840c299d08ce9433d6c85/roles/67e4864efce9469bb6bdcca8ee5d5b24 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:51.085257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5f1ca-2376-4241-aed5-d506e1c6e975 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:51.175859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:51.193867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9217945-d1f7-4cda-b39f-892eca73a877 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:51.217566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5f1ca-2376-4241-aed5-d506e1c6e975 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:51.243680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:51.261312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03139d-c290-4e0a-a03b-8a9dc2b0ef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:51.262933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03139d-c290-4e0a-a03b-8a9dc2b0ef28 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:51.263643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03139d-c290-4e0a-a03b-8a9dc2b0ef28 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:51.263843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a36593-06af-45d2-a216-5972c87bec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:51.264394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03139d-c290-4e0a-a03b-8a9dc2b0ef28 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:51.264647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36593-06af-45d2-a216-5972c87bec58 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:51.264933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36593-06af-45d2-a216-5972c87bec58 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:51.265211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a36593-06af-45d2-a216-5972c87bec58 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:51.266114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03139d-c290-4e0a-a03b-8a9dc2b0ef28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:51.266639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03139d-c290-4e0a-a03b-8a9dc2b0ef28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:51.268389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03139d-c290-4e0a-a03b-8a9dc2b0ef28 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:51.287096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a36593-06af-45d2-a216-5972c87bec58 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:51.287096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a36593-06af-45d2-a216-5972c87bec58 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:51.287666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 292/593] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:51.289576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03139d-c290-4e0a-a03b-8a9dc2b0ef28 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-d3a36593-06af-45d2-a216-5972c87bec58 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:51.316701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:51.317714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:51.318670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:51.318670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:51.318670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:51.318670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:51.318670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:51.318670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:51.318670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:51.318670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:51.318670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:00:51.318670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:51.319855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 302/594] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:51.325526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a36593-06af-45d2-a216-5972c87bec58 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:51.424713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03139d-c290-4e0a-a03b-8a9dc2b0ef28 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:51.435425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a36593-06af-45d2-a216-5972c87bec58 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:51.463772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a757904-4985-4d4c-b8dc-bcc031120ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:51.465501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a757904-4985-4d4c-b8dc-bcc031120ddd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:51.465997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a757904-4985-4d4c-b8dc-bcc031120ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:51.466482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a757904-4985-4d4c-b8dc-bcc031120ddd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:51.472067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc6b3e-a30f-4a69-a504-69558493f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:51.472839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc6b3e-a30f-4a69-a504-69558493f86b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:51.473193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc6b3e-a30f-4a69-a504-69558493f86b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:51.473540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc6b3e-a30f-4a69-a504-69558493f86b None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:51.474462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc6b3e-a30f-4a69-a504-69558493f86b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:51.474845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc6b3e-a30f-4a69-a504-69558493f86b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:51.476395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fc6b3e-a30f-4a69-a504-69558493f86b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:51.476843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a757904-4985-4d4c-b8dc-bcc031120ddd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:51.478594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a757904-4985-4d4c-b8dc-bcc031120ddd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8a757904-4985-4d4c-b8dc-bcc031120ddd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:51.494020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:51.495317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:51.497056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:51.497056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:51.497056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:51.497056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:51.497056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:51.497056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:51.497056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:51.497056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:51.497056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:00:51.497056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:51.497598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 303/595] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 5 15:00:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:51.503209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc6b3e-a30f-4a69-a504-69558493f86b None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:51.503339 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 293/596] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:51.506387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a757904-4985-4d4c-b8dc-bcc031120ddd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:51.513221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:51.548238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:51.611555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc6b3e-a30f-4a69-a504-69558493f86b None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:51.631474 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a757904-4985-4d4c-b8dc-bcc031120ddd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:51.647478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa0198-002b-40fe-8532-920bab8defb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:51.648474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa0198-002b-40fe-8532-920bab8defb8 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:51.648683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa0198-002b-40fe-8532-920bab8defb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:51.648924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa0198-002b-40fe-8532-920bab8defb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:51.660342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa0198-002b-40fe-8532-920bab8defb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:51.661805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa0198-002b-40fe-8532-920bab8defb8 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:51.671719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4de61c-4147-4433-a75f-ff361148a54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:51.672727 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de61c-4147-4433-a75f-ff361148a54f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:51.673233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de61c-4147-4433-a75f-ff361148a54f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:51.673718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4de61c-4147-4433-a75f-ff361148a54f None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:51.674841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4de61c-4147-4433-a75f-ff361148a54f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:51.675391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4de61c-4147-4433-a75f-ff361148a54f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:51.677118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4de61c-4147-4433-a75f-ff361148a54f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-1dfa0198-002b-40fe-8532-920bab8defb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:51.680852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:51.681748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:51.683251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:51.683251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:51.683251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:51.683251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:51.683251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 304/597] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:51.687672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa0198-002b-40fe-8532-920bab8defb8 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:51.707242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 294/598] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Jul 5 15:00:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:51.709403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4de61c-4147-4433-a75f-ff361148a54f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:51.749304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:51.794929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa0198-002b-40fe-8532-920bab8defb8 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:51.813561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:51.824632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4de61c-4147-4433-a75f-ff361148a54f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:51.835798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd74b5b-fed4-481f-8491-40f07f6033e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:51.836705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd74b5b-fed4-481f-8491-40f07f6033e9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:51.837132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd74b5b-fed4-481f-8491-40f07f6033e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:51.837549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd74b5b-fed4-481f-8491-40f07f6033e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:51.838454 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd74b5b-fed4-481f-8491-40f07f6033e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:51.838915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd74b5b-fed4-481f-8491-40f07f6033e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:51.840501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd74b5b-fed4-481f-8491-40f07f6033e9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:51.852955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 305/599] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:51.864987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd74b5b-fed4-481f-8491-40f07f6033e9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:51.906645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77384d9f-fce8-49b9-a0a6-7afe8d50e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:51.907719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77384d9f-fce8-49b9-a0a6-7afe8d50e443 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:51.908147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77384d9f-fce8-49b9-a0a6-7afe8d50e443 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:51.908567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77384d9f-fce8-49b9-a0a6-7afe8d50e443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:51.929145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77384d9f-fce8-49b9-a0a6-7afe8d50e443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:51.933105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77384d9f-fce8-49b9-a0a6-7afe8d50e443 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-77384d9f-fce8-49b9-a0a6-7afe8d50e443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:51.972688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:51.973801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:51.974883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:51.974883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:51.974883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:51.974883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:51.979077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 295/600] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:51.984479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77384d9f-fce8-49b9-a0a6-7afe8d50e443 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:52.062320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd74b5b-fed4-481f-8491-40f07f6033e9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:52.099324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:52.132448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d497-8fbb-4c43-9495-79688f5ca858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:52.133595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d497-8fbb-4c43-9495-79688f5ca858 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:52.135097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d497-8fbb-4c43-9495-79688f5ca858 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:52.135720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d497-8fbb-4c43-9495-79688f5ca858 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:52.147916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300d497-8fbb-4c43-9495-79688f5ca858 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:52.150331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300d497-8fbb-4c43-9495-79688f5ca858 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-6300d497-8fbb-4c43-9495-79688f5ca858 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:52.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:52.177542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:52.178621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:52.178621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:52.178621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:52.178621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:52.178621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:52.178621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:52.178621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:52.178621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:52.182759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77384d9f-fce8-49b9-a0a6-7afe8d50e443 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:52.185375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 306/601] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:52.200208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d497-8fbb-4c43-9495-79688f5ca858 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:52.255443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d8dd8-36d5-4b9c-83a9-e7aba862cf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:52.255443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d8dd8-36d5-4b9c-83a9-e7aba862cf4f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:52.255443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d8dd8-36d5-4b9c-83a9-e7aba862cf4f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:52.255443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d8dd8-36d5-4b9c-83a9-e7aba862cf4f None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:52.255929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d8dd8-36d5-4b9c-83a9-e7aba862cf4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:52.255929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d8dd8-36d5-4b9c-83a9-e7aba862cf4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:52.255929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d8dd8-36d5-4b9c-83a9-e7aba862cf4f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:52.269844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 296/602] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:52.271266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d8dd8-36d5-4b9c-83a9-e7aba862cf4f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:52.275000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:52.330366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d497-8fbb-4c43-9495-79688f5ca858 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:52.400298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d8dd8-36d5-4b9c-83a9-e7aba862cf4f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:52.431762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef76a32-4c5c-4a1c-a7a3-c8730c2eab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:52.432448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef76a32-4c5c-4a1c-a7a3-c8730c2eab08 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:52.432738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef76a32-4c5c-4a1c-a7a3-c8730c2eab08 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:52.433072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef76a32-4c5c-4a1c-a7a3-c8730c2eab08 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:52.434361 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef76a32-4c5c-4a1c-a7a3-c8730c2eab08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:52.434677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef76a32-4c5c-4a1c-a7a3-c8730c2eab08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:52.436162 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef76a32-4c5c-4a1c-a7a3-c8730c2eab08 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:52.438330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a279a-7242-40d6-94a5-95c287104dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:52.439202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a279a-7242-40d6-94a5-95c287104dad None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:52.439493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a279a-7242-40d6-94a5-95c287104dad None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:52.439769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a279a-7242-40d6-94a5-95c287104dad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:52.446909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a279a-7242-40d6-94a5-95c287104dad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:52.448409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a279a-7242-40d6-94a5-95c287104dad None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:52.462435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 307/603] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-5b6a279a-7242-40d6-94a5-95c287104dad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:52.463469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:52.465315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:52.466205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:52.466205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:52.466205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:52.466205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:52.466205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:52.466205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:52.466205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:52.466205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:52.469849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 297/604] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:52.471171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef76a32-4c5c-4a1c-a7a3-c8730c2eab08 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:52.472102 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a279a-7242-40d6-94a5-95c287104dad None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:52.510681 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:52.561000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:52.621112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef76a32-4c5c-4a1c-a7a3-c8730c2eab08 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:52.635664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a279a-7242-40d6-94a5-95c287104dad None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:52.688945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf79eb-a860-46c3-a7ab-0fba143458ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:52.690570 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf79eb-a860-46c3-a7ab-0fba143458ef None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:52.692048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf79eb-a860-46c3-a7ab-0fba143458ef None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:52.692480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf79eb-a860-46c3-a7ab-0fba143458ef None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:52.693646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf79eb-a860-46c3-a7ab-0fba143458ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:52.695538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf79eb-a860-46c3-a7ab-0fba143458ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:52.697807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbf79eb-a860-46c3-a7ab-0fba143458ef None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:52.704332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf7f15-3919-4d05-bf96-9d49fc66a312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:52.705502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf7f15-3919-4d05-bf96-9d49fc66a312 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:52.706048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf7f15-3919-4d05-bf96-9d49fc66a312 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:52.706498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf7f15-3919-4d05-bf96-9d49fc66a312 None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/8ad7cee9ffed415b9a30176b1888e562/roles/67e4864efce9469bb6bdcca8ee5d5b24` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:52.722210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf7f15-3919-4d05-bf96-9d49fc66a312 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=8ad7cee9ffed415b9a30176b1888e562, role_id=67e4864efce9469bb6bdcca8ee5d5b24)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:52.723958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf7f15-3919-4d05-bf96-9d49fc66a312 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:52.729586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 308/605] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:52.734653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf79eb-a860-46c3-a7ab-0fba143458ef None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:52.749106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 298/606] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:00:52 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/8ad7cee9ffed415b9a30176b1888e562/roles/67e4864efce9469bb6bdcca8ee5d5b24 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:00:52.879603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:52.887605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf79eb-a860-46c3-a7ab-0fba143458ef None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:52.930487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a4534-fca9-492c-9b61-e554e58e2c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:52.931717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a4534-fca9-492c-9b61-e554e58e2c73 None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:52.932181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a4534-fca9-492c-9b61-e554e58e2c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:52.932623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a4534-fca9-492c-9b61-e554e58e2c73 None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/cd7fa93466c54632ad95834297410e00/roles/67e4864efce9469bb6bdcca8ee5d5b24` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:52.955660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a4534-fca9-492c-9b61-e554e58e2c73 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=cd7fa93466c54632ad95834297410e00, role_id=67e4864efce9469bb6bdcca8ee5d5b24)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:52.957386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a4534-fca9-492c-9b61-e554e58e2c73 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:52.979160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 309/607] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:00:52 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/cd7fa93466c54632ad95834297410e00/roles/67e4864efce9469bb6bdcca8ee5d5b24 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:00:53.155104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0b69-63eb-47de-a986-9e663481f7a6 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:53.155637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0b69-63eb-47de-a986-9e663481f7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:53.156073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff0b69-63eb-47de-a986-9e663481f7a6 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:53.157593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 299/608] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:53.163055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3699b-ceee-4dc8-b84b-dda494aeb408 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:53.163356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3699b-ceee-4dc8-b84b-dda494aeb408 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:53.163645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b3699b-ceee-4dc8-b84b-dda494aeb408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:53.182115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-21b3699b-ceee-4dc8-b84b-dda494aeb408 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:53.183886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-21b3699b-ceee-4dc8-b84b-dda494aeb408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:53.262856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 310/609] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:53.297467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff0b69-63eb-47de-a986-9e663481f7a6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:53.336610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121abc35-7032-444b-985c-e38fc07ff2b7 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:53.337052 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121abc35-7032-444b-985c-e38fc07ff2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:53.337442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121abc35-7032-444b-985c-e38fc07ff2b7 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:53.339203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 311/610] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:53.344290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4882-576a-4b95-a4ff-8502257cc2be None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:53.344688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4882-576a-4b95-a4ff-8502257cc2be None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:53.345078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c4882-576a-4b95-a4ff-8502257cc2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:53.369320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-9f1c4882-576a-4b95-a4ff-8502257cc2be None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:53.371628 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9f1c4882-576a-4b95-a4ff-8502257cc2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:53.473948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff0b69-63eb-47de-a986-9e663481f7a6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:53.503412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:53.510649 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 312/611] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:53.525659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4e38a-b69a-4dee-9972-1735a0988fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:53.526395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4e38a-b69a-4dee-9972-1735a0988fdb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:53.526677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4e38a-b69a-4dee-9972-1735a0988fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:53.526953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4e38a-b69a-4dee-9972-1735a0988fdb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:53.538367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4e38a-b69a-4dee-9972-1735a0988fdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:53.545017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c4882-576a-4b95-a4ff-8502257cc2be None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4ee4e38a-b69a-4dee-9972-1735a0988fdb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:53.554351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:53.555390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:53.556267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:53.557795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 300/612] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:53.558960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:53.564915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4e38a-b69a-4dee-9972-1735a0988fdb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:53.678698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4e38a-b69a-4dee-9972-1735a0988fdb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:53.686204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c4882-576a-4b95-a4ff-8502257cc2be None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:53.723942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834159fb-26d1-4626-aea4-5e3da489e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:53.724905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834159fb-26d1-4626-aea4-5e3da489e947 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:53.725197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834159fb-26d1-4626-aea4-5e3da489e947 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:53.725510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834159fb-26d1-4626-aea4-5e3da489e947 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:53.726907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834159fb-26d1-4626-aea4-5e3da489e947 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:53.727333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834159fb-26d1-4626-aea4-5e3da489e947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:53.729232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834159fb-26d1-4626-aea4-5e3da489e947 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:53.739485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f5c50-a7b3-41f5-ae9b-1a90f929bfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:53.740237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 301/613] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:53.741602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5891e-b935-4e19-9d26-a3db5d5f7981 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:53.741899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5891e-b935-4e19-9d26-a3db5d5f7981 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:53.742184 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5891e-b935-4e19-9d26-a3db5d5f7981 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:53.743618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 302/614] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:53.744884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f5c50-a7b3-41f5-ae9b-1a90f929bfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:53.745290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f5c50-a7b3-41f5-ae9b-1a90f929bfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:53.745670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f5c50-a7b3-41f5-ae9b-1a90f929bfe4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:53.748515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5891e-b935-4e19-9d26-a3db5d5f7981 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:53.759029 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f5c50-a7b3-41f5-ae9b-1a90f929bfe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-f34f5c50-a7b3-41f5-ae9b-1a90f929bfe4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:53.770263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:53.771279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:53.772124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:53.773914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 313/615] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:53.773914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99d949-78a3-4a9f-bfcf-ef2052263253 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:53.775073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99d949-78a3-4a9f-bfcf-ef2052263253 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:53.775073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce99d949-78a3-4a9f-bfcf-ef2052263253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:53.791345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:53.798108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ce99d949-78a3-4a9f-bfcf-ef2052263253 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:53.799560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ce99d949-78a3-4a9f-bfcf-ef2052263253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:53.864308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:53.877781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5891e-b935-4e19-9d26-a3db5d5f7981 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:53.879313 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 314/616] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:53.880222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99d949-78a3-4a9f-bfcf-ef2052263253 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:53.920572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cd9f4-c264-42eb-b807-7abf16e9b2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:53.922780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cd9f4-c264-42eb-b807-7abf16e9b2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:53.923770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cd9f4-c264-42eb-b807-7abf16e9b2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:53.924529 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cd9f4-c264-42eb-b807-7abf16e9b2e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:53.938829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cd9f4-c264-42eb-b807-7abf16e9b2e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:53.941234 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cd9f4-c264-42eb-b807-7abf16e9b2e4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-a06cd9f4-c264-42eb-b807-7abf16e9b2e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:53.965791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:53.966667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:53.967629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:53.967629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:53.967629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:53.967629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:53.967629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 303/617] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:53.973198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cd9f4-c264-42eb-b807-7abf16e9b2e4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.007266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce99d949-78a3-4a9f-bfcf-ef2052263253 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.043424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4c49c-5548-4f40-b6a3-3d8c8072e807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.044035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4c49c-5548-4f40-b6a3-3d8c8072e807 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.044595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4c49c-5548-4f40-b6a3-3d8c8072e807 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.044723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4c49c-5548-4f40-b6a3-3d8c8072e807 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.045487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4c49c-5548-4f40-b6a3-3d8c8072e807 None admin] RBAC: Adding query filter params (name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:54.045743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4c49c-5548-4f40-b6a3-3d8c8072e807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:54.047176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4c49c-5548-4f40-b6a3-3d8c8072e807 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:54.055983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 315/618] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:54.056789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4c49c-5548-4f40-b6a3-3d8c8072e807 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.072427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:54.087544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cd9f4-c264-42eb-b807-7abf16e9b2e4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.123250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:54.124027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebee1c-7a46-4999-bd2b-f3e0c5edd9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.124828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebee1c-7a46-4999-bd2b-f3e0c5edd9bb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.125204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebee1c-7a46-4999-bd2b-f3e0c5edd9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.125702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebee1c-7a46-4999-bd2b-f3e0c5edd9bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.138521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebee1c-7a46-4999-bd2b-f3e0c5edd9bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b4ebee1c-7a46-4999-bd2b-f3e0c5edd9bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:54.146713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:54.147878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:54.148731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:54.154377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 304/619] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:54.155398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebee1c-7a46-4999-bd2b-f3e0c5edd9bb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.172916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4c49c-5548-4f40-b6a3-3d8c8072e807 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.213018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c9586-78d6-4e56-bbbc-2054648e954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.214499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c9586-78d6-4e56-bbbc-2054648e954b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.215652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c9586-78d6-4e56-bbbc-2054648e954b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.216115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c9586-78d6-4e56-bbbc-2054648e954b None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.217140 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c9586-78d6-4e56-bbbc-2054648e954b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:54.217678 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c9586-78d6-4e56-bbbc-2054648e954b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:54.219423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c9586-78d6-4e56-bbbc-2054648e954b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:54.232997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 316/620] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:54.234323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c9586-78d6-4e56-bbbc-2054648e954b None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.268227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebee1c-7a46-4999-bd2b-f3e0c5edd9bb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.305842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e43860-cde9-4cc0-b39c-1712389c58d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.306550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e43860-cde9-4cc0-b39c-1712389c58d1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.306832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e43860-cde9-4cc0-b39c-1712389c58d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.307131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e43860-cde9-4cc0-b39c-1712389c58d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.323281 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e43860-cde9-4cc0-b39c-1712389c58d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:54.324832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e43860-cde9-4cc0-b39c-1712389c58d1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:54.333824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-97e43860-cde9-4cc0-b39c-1712389c58d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:54.345453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:54.346527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:54.347471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:54.347471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:54.347471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:54.347471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:54.347471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 305/621] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:54.348177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e43860-cde9-4cc0-b39c-1712389c58d1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.350222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c9586-78d6-4e56-bbbc-2054648e954b None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.372800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:54.388732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db5a2f-8a82-4601-958c-ad484a68d4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.389540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db5a2f-8a82-4601-958c-ad484a68d4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.389927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db5a2f-8a82-4601-958c-ad484a68d4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.390325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db5a2f-8a82-4601-958c-ad484a68d4c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.391229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db5a2f-8a82-4601-958c-ad484a68d4c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:54.391637 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db5a2f-8a82-4601-958c-ad484a68d4c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:54.393205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db5a2f-8a82-4601-958c-ad484a68d4c0 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:54.406938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 317/622] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:54.406938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db5a2f-8a82-4601-958c-ad484a68d4c0 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.465964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e43860-cde9-4cc0-b39c-1712389c58d1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.500944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce11e1-40ae-4197-b276-6c746b94b21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.502478 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce11e1-40ae-4197-b276-6c746b94b21d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.502924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce11e1-40ae-4197-b276-6c746b94b21d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.503436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce11e1-40ae-4197-b276-6c746b94b21d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.509235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db5a2f-8a82-4601-958c-ad484a68d4c0 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.513161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce11e1-40ae-4197-b276-6c746b94b21d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:54.514598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce11e1-40ae-4197-b276-6c746b94b21d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-5cce11e1-40ae-4197-b276-6c746b94b21d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:54.529842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:54.530836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:54.532111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 306/623] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:54.532882 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce11e1-40ae-4197-b276-6c746b94b21d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.557487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05dad54-0503-4a98-ab9d-ffcfb2b9f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.558394 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05dad54-0503-4a98-ab9d-ffcfb2b9f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.558989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05dad54-0503-4a98-ab9d-ffcfb2b9f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.559743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05dad54-0503-4a98-ab9d-ffcfb2b9f0c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.560909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05dad54-0503-4a98-ab9d-ffcfb2b9f0c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:54.561376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05dad54-0503-4a98-ab9d-ffcfb2b9f0c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:54.563565 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05dad54-0503-4a98-ab9d-ffcfb2b9f0c9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:54.579350 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 318/624] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:54.580418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05dad54-0503-4a98-ab9d-ffcfb2b9f0c9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.599449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:54.642266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:54.644012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce11e1-40ae-4197-b276-6c746b94b21d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.692117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15368c-b967-4bc1-bef6-1be812818241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.693511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15368c-b967-4bc1-bef6-1be812818241 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.693996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15368c-b967-4bc1-bef6-1be812818241 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.694814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c15368c-b967-4bc1-bef6-1be812818241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.703603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05dad54-0503-4a98-ab9d-ffcfb2b9f0c9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.710228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15368c-b967-4bc1-bef6-1be812818241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:54.712564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c15368c-b967-4bc1-bef6-1be812818241 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4c15368c-b967-4bc1-bef6-1be812818241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:54.737674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:54.738631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:54.739577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:54.739577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:54.739577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:54.739577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:54.743394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 307/625] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:54.744525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15368c-b967-4bc1-bef6-1be812818241 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.748969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09c747-a771-4655-b6cd-e8e5e1ae224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.750148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09c747-a771-4655-b6cd-e8e5e1ae224a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.750148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09c747-a771-4655-b6cd-e8e5e1ae224a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.750328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09c747-a771-4655-b6cd-e8e5e1ae224a None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.751243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a09c747-a771-4655-b6cd-e8e5e1ae224a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:54.751623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a09c747-a771-4655-b6cd-e8e5e1ae224a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:54.753140 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a09c747-a771-4655-b6cd-e8e5e1ae224a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:54.771899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 319/626] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:54.772948 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09c747-a771-4655-b6cd-e8e5e1ae224a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.870620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c15368c-b967-4bc1-bef6-1be812818241 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.884761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:54.885395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:54.891262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09c747-a771-4655-b6cd-e8e5e1ae224a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:54.913823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec650ec-9b2a-4c27-a940-9c232c08a009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.915075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec650ec-9b2a-4c27-a940-9c232c08a009 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.915567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec650ec-9b2a-4c27-a940-9c232c08a009 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.916117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec650ec-9b2a-4c27-a940-9c232c08a009 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.925220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec650ec-9b2a-4c27-a940-9c232c08a009 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:54.927208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec650ec-9b2a-4c27-a940-9c232c08a009 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:54.936012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b6cd2-f76b-496f-a07c-a8d0b3e864a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:54.936689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b6cd2-f76b-496f-a07c-a8d0b3e864a5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:54.936979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b6cd2-f76b-496f-a07c-a8d0b3e864a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:54.937263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b6cd2-f76b-496f-a07c-a8d0b3e864a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:54.938004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b6cd2-f76b-496f-a07c-a8d0b3e864a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:54.938318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b6cd2-f76b-496f-a07c-a8d0b3e864a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:54.939733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b6cd2-f76b-496f-a07c-a8d0b3e864a5 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-cec650ec-9b2a-4c27-a940-9c232c08a009 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:54.944510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:54.945417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:54.946347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:54.946347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:54.946347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:54.946347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:54.946347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:54.946347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:54.946347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:54.946347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:54.946347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:00:54.946347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:54.949681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 308/627] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:54.950732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec650ec-9b2a-4c27-a940-9c232c08a009 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:54.956849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 320/628] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:54.956849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b6cd2-f76b-496f-a07c-a8d0b3e864a5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:55.064523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec650ec-9b2a-4c27-a940-9c232c08a009 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:55.095463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b6cd2-f76b-496f-a07c-a8d0b3e864a5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:55.101623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c039ef-64dd-41ac-997f-d04a8ceb3493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:55.102478 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c039ef-64dd-41ac-997f-d04a8ceb3493 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.102908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c039ef-64dd-41ac-997f-d04a8ceb3493 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.103362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c039ef-64dd-41ac-997f-d04a8ceb3493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.141107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad56340-ac51-45d8-9fcc-e91eff80b1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:55.142211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad56340-ac51-45d8-9fcc-e91eff80b1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.143230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad56340-ac51-45d8-9fcc-e91eff80b1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.143699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad56340-ac51-45d8-9fcc-e91eff80b1c4 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.144565 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad56340-ac51-45d8-9fcc-e91eff80b1c4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:55.144881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad56340-ac51-45d8-9fcc-e91eff80b1c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:55.146322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad56340-ac51-45d8-9fcc-e91eff80b1c4 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:55.155415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:55.166375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 321/629] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:55.167286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad56340-ac51-45d8-9fcc-e91eff80b1c4 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:55.187105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:55.190161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c039ef-64dd-41ac-997f-d04a8ceb3493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:55.193933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c039ef-64dd-41ac-997f-d04a8ceb3493 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:55.269742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad56340-ac51-45d8-9fcc-e91eff80b1c4 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:55.295933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 309/630] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:55.296846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c039ef-64dd-41ac-997f-d04a8ceb3493 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:55.304545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b098-3b2e-41c8-8c0b-4b880980bb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:55.305415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b098-3b2e-41c8-8c0b-4b880980bb59 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.305707 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b098-3b2e-41c8-8c0b-4b880980bb59 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.305988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b098-3b2e-41c8-8c0b-4b880980bb59 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.313182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3b098-3b2e-41c8-8c0b-4b880980bb59 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:55.314639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3b098-3b2e-41c8-8c0b-4b880980bb59 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-82d3b098-3b2e-41c8-8c0b-4b880980bb59 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:55.329886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:55.330869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:55.332473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 322/631] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:55.333077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65ea96-1d04-473c-b609-a7814f207748 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.333077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65ea96-1d04-473c-b609-a7814f207748 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.333077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65ea96-1d04-473c-b609-a7814f207748 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.334623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 323/632] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:55.335678 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b5913-3629-4bda-ae07-ade42f85ec47 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.335964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b5913-3629-4bda-ae07-ade42f85ec47 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.336240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b5913-3629-4bda-ae07-ade42f85ec47 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.337564 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 324/633] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:55.340672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b5913-3629-4bda-ae07-ade42f85ec47 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:55.365475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:55.394550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c039ef-64dd-41ac-997f-d04a8ceb3493 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:55.418480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:55.429173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ef149-03b9-4ff3-be4c-11f8e81cb3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:55.430008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ef149-03b9-4ff3-be4c-11f8e81cb3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.430296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ef149-03b9-4ff3-be4c-11f8e81cb3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.430573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ef149-03b9-4ff3-be4c-11f8e81cb3b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.440322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ef149-03b9-4ff3-be4c-11f8e81cb3b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:55.441671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b5913-3629-4bda-ae07-ade42f85ec47 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:55.441796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ef149-03b9-4ff3-be4c-11f8e81cb3b7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4b7ef149-03b9-4ff3-be4c-11f8e81cb3b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:55.459537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:55.460580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:55.466695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:55.466695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:55.466695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:55.466695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:55.466695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 310/634] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:55.466695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ef149-03b9-4ff3-be4c-11f8e81cb3b7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:55.482051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc78a8a-6dd7-4e66-8dcd-bb84db12fc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:55.482854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc78a8a-6dd7-4e66-8dcd-bb84db12fc4b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.483166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc78a8a-6dd7-4e66-8dcd-bb84db12fc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.483444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc78a8a-6dd7-4e66-8dcd-bb84db12fc4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.493357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc78a8a-6dd7-4e66-8dcd-bb84db12fc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:55.494812 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc78a8a-6dd7-4e66-8dcd-bb84db12fc4b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-4fc78a8a-6dd7-4e66-8dcd-bb84db12fc4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:55.514172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:55.515106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:55.516000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:55.516000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:55.516000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:55.516000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:55.516000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 325/635] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:55.516689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1f99f-4a50-47c9-a7a9-e484bcf7bfb9 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.516836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1f99f-4a50-47c9-a7a9-e484bcf7bfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.517113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a1f99f-4a50-47c9-a7a9-e484bcf7bfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.533133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-56a1f99f-4a50-47c9-a7a9-e484bcf7bfb9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:55.534316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-56a1f99f-4a50-47c9-a7a9-e484bcf7bfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:55.565295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ef149-03b9-4ff3-be4c-11f8e81cb3b7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:55.588682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:55.599086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 326/636] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:55.599973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b153-9e61-44bd-8d5f-a5ff79008b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:55.600427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76439-3be7-4b4e-9428-450dcd33f3ea None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.600639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b153-9e61-44bd-8d5f-a5ff79008b79 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.600759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76439-3be7-4b4e-9428-450dcd33f3ea None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.600929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b153-9e61-44bd-8d5f-a5ff79008b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.601025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e76439-3be7-4b4e-9428-450dcd33f3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.601187 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2b153-9e61-44bd-8d5f-a5ff79008b79 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.602022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b153-9e61-44bd-8d5f-a5ff79008b79 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:55.602337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b153-9e61-44bd-8d5f-a5ff79008b79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:55.603788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2b153-9e61-44bd-8d5f-a5ff79008b79 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:55.617486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-16e76439-3be7-4b4e-9428-450dcd33f3ea None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:55.619204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-16e76439-3be7-4b4e-9428-450dcd33f3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:55.621694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 311/637] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 5 15:00:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:55.622504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b153-9e61-44bd-8d5f-a5ff79008b79 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:55.674533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:55.685815 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 327/638] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:00:55.686706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e76439-3be7-4b4e-9428-450dcd33f3ea None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:55.723788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2b153-9e61-44bd-8d5f-a5ff79008b79 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:55.757149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7259631-f24b-4f00-8d29-3995640098a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:55.757813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7259631-f24b-4f00-8d29-3995640098a0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.758098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7259631-f24b-4f00-8d29-3995640098a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.758378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7259631-f24b-4f00-8d29-3995640098a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.759129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7259631-f24b-4f00-8d29-3995640098a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:55.759441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7259631-f24b-4f00-8d29-3995640098a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:55.760800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7259631-f24b-4f00-8d29-3995640098a0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:55.772469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 312/639] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:55.773242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7259631-f24b-4f00-8d29-3995640098a0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:55.786565 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e76439-3be7-4b4e-9428-450dcd33f3ea None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:55.814589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:55.821775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c9f80-5789-45e3-b670-45844618fbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:55.822439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c9f80-5789-45e3-b670-45844618fbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.822730 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c9f80-5789-45e3-b670-45844618fbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.823013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724c9f80-5789-45e3-b670-45844618fbb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.823787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c9f80-5789-45e3-b670-45844618fbb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:55.824103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c9f80-5789-45e3-b670-45844618fbb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:55.825478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724c9f80-5789-45e3-b670-45844618fbb9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:55.836855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 328/640] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:55.837639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c9f80-5789-45e3-b670-45844618fbb9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:55.878232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7259631-f24b-4f00-8d29-3995640098a0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:55.886393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:55.911777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cbb64-a431-416b-ae8a-7e9f139f03b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:55.912572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cbb64-a431-416b-ae8a-7e9f139f03b8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.912854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cbb64-a431-416b-ae8a-7e9f139f03b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.913128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cbb64-a431-416b-ae8a-7e9f139f03b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:55.924013 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cbb64-a431-416b-ae8a-7e9f139f03b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-810cbb64-a431-416b-ae8a-7e9f139f03b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:55.931923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:55.932959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 313/641] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:55.933850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cbb64-a431-416b-ae8a-7e9f139f03b8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:55.939109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724c9f80-5789-45e3-b670-45844618fbb9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:55.972415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a657845-0247-40ce-a6b9-c3c90fc085cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:55.973061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a657845-0247-40ce-a6b9-c3c90fc085cd None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:55.973346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a657845-0247-40ce-a6b9-c3c90fc085cd None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:55.973624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a657845-0247-40ce-a6b9-c3c90fc085cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.034431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cbb64-a431-416b-ae8a-7e9f139f03b8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.035930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:56.041699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a657845-0247-40ce-a6b9-c3c90fc085cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.043256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a657845-0247-40ce-a6b9-c3c90fc085cd None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.068858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aaab67-6c39-4959-b36b-536496330134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.069731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaab67-6c39-4959-b36b-536496330134 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.070077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaab67-6c39-4959-b36b-536496330134 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.070358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aaab67-6c39-4959-b36b-536496330134 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.080631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aaab67-6c39-4959-b36b-536496330134 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d0aaab67-6c39-4959-b36b-536496330134 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:56.088232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:56.089304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:56.090723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:56.091395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:56.095010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 314/642] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:56.095918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aaab67-6c39-4959-b36b-536496330134 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.141361 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 329/643] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:56.142192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a657845-0247-40ce-a6b9-c3c90fc085cd None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.206604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aaab67-6c39-4959-b36b-536496330134 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.237916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a657845-0247-40ce-a6b9-c3c90fc085cd None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.239722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dffd9b-b34a-4530-a1fb-bdf1164d2265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.240627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dffd9b-b34a-4530-a1fb-bdf1164d2265 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.240912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dffd9b-b34a-4530-a1fb-bdf1164d2265 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.241194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dffd9b-b34a-4530-a1fb-bdf1164d2265 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.247955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dffd9b-b34a-4530-a1fb-bdf1164d2265 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.249416 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dffd9b-b34a-4530-a1fb-bdf1164d2265 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d0dffd9b-b34a-4530-a1fb-bdf1164d2265 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:56.263646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:56.264717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:56.266088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:56.266088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:56.266088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:56.266088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:56.266088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:56.266088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:56.266088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:56.266088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:56.266088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 315/644] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:56.266088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dffd9b-b34a-4530-a1fb-bdf1164d2265 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.273545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f41add-9181-4b7d-adc0-b0e0d92729b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.274338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f41add-9181-4b7d-adc0-b0e0d92729b5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.274622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f41add-9181-4b7d-adc0-b0e0d92729b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.274897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f41add-9181-4b7d-adc0-b0e0d92729b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.281627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f41add-9181-4b7d-adc0-b0e0d92729b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.283086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f41add-9181-4b7d-adc0-b0e0d92729b5 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.291530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-78f41add-9181-4b7d-adc0-b0e0d92729b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:56.296932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:56.298002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:56.299211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:56.299211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:56.299211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:56.299211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:56.299211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:56.299211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:56.299211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:56.299211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:56.299211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 330/645] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:56.299211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f41add-9181-4b7d-adc0-b0e0d92729b5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.310998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:56.369650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dffd9b-b34a-4530-a1fb-bdf1164d2265 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.397424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f41add-9181-4b7d-adc0-b0e0d92729b5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.402999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445319b7-d4e3-4ff1-a4b4-b18d97118d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.403695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445319b7-d4e3-4ff1-a4b4-b18d97118d0b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.403982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445319b7-d4e3-4ff1-a4b4-b18d97118d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.404262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445319b7-d4e3-4ff1-a4b4-b18d97118d0b None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.405006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445319b7-d4e3-4ff1-a4b4-b18d97118d0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:56.405313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445319b7-d4e3-4ff1-a4b4-b18d97118d0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.406698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445319b7-d4e3-4ff1-a4b4-b18d97118d0b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.414445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 316/646] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:56.415236 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445319b7-d4e3-4ff1-a4b4-b18d97118d0b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.430915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0973623-549d-40b2-ac94-c9bb2af1aa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.431654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0973623-549d-40b2-ac94-c9bb2af1aa2a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.431961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0973623-549d-40b2-ac94-c9bb2af1aa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.432251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0973623-549d-40b2-ac94-c9bb2af1aa2a None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.433013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0973623-549d-40b2-ac94-c9bb2af1aa2a None admin] RBAC: Adding query filter params (name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:56.433330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0973623-549d-40b2-ac94-c9bb2af1aa2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.434705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0973623-549d-40b2-ac94-c9bb2af1aa2a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.442578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 331/647] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:56.443376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0973623-549d-40b2-ac94-c9bb2af1aa2a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.497147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:56.511321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:56.513873 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445319b7-d4e3-4ff1-a4b4-b18d97118d0b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.539834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0973623-549d-40b2-ac94-c9bb2af1aa2a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.546627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e857a5-0a95-44e9-ab87-ced2480c9685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.547445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e857a5-0a95-44e9-ab87-ced2480c9685 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.547730 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e857a5-0a95-44e9-ab87-ced2480c9685 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.548008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e857a5-0a95-44e9-ab87-ced2480c9685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.558231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e857a5-0a95-44e9-ab87-ced2480c9685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.560244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e857a5-0a95-44e9-ab87-ced2480c9685 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.572725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08816d-359b-4cda-9e01-83556050a356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.573380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08816d-359b-4cda-9e01-83556050a356 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.573668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08816d-359b-4cda-9e01-83556050a356 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.573996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08816d-359b-4cda-9e01-83556050a356 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.574864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08816d-359b-4cda-9e01-83556050a356 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:56.575216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08816d-359b-4cda-9e01-83556050a356 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.576905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08816d-359b-4cda-9e01-83556050a356 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-86e857a5-0a95-44e9-ab87-ced2480c9685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:56.577232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:56.578432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:56.579345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:56.579345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:56.579345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:56.579345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:56.579345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 317/648] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:56.579637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e857a5-0a95-44e9-ab87-ced2480c9685 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.592572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 332/649] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:56.593442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08816d-359b-4cda-9e01-83556050a356 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.677494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e857a5-0a95-44e9-ab87-ced2480c9685 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.690237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08816d-359b-4cda-9e01-83556050a356 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.710606 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d5960-cc20-4304-b90e-20b6deaeb572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.711286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d5960-cc20-4304-b90e-20b6deaeb572 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.711598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d5960-cc20-4304-b90e-20b6deaeb572 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.711874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d5960-cc20-4304-b90e-20b6deaeb572 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.712664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d5960-cc20-4304-b90e-20b6deaeb572 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:56.712978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d5960-cc20-4304-b90e-20b6deaeb572 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.714397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d5960-cc20-4304-b90e-20b6deaeb572 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.716815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:56.719957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:56.725392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae8c32-19d3-4d2c-a30a-ac914d1a27f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.726185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae8c32-19d3-4d2c-a30a-ac914d1a27f9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.726476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae8c32-19d3-4d2c-a30a-ac914d1a27f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.727013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae8c32-19d3-4d2c-a30a-ac914d1a27f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.727143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 318/650] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:56.727651 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d5960-cc20-4304-b90e-20b6deaeb572 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.737276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae8c32-19d3-4d2c-a30a-ac914d1a27f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.738732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae8c32-19d3-4d2c-a30a-ac914d1a27f9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-2bae8c32-19d3-4d2c-a30a-ac914d1a27f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:56.757296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:56.758388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:56.759548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:56.759548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:56.759548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:56.759548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:56.759548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 333/651] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:56.759548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae8c32-19d3-4d2c-a30a-ac914d1a27f9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.830173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d5960-cc20-4304-b90e-20b6deaeb572 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.856375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae8c32-19d3-4d2c-a30a-ac914d1a27f9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:56.863539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e68cc-a9ef-4d1b-855b-b8dc3cb89aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.864327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e68cc-a9ef-4d1b-855b-b8dc3cb89aff None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.864612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e68cc-a9ef-4d1b-855b-b8dc3cb89aff None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.864887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e68cc-a9ef-4d1b-855b-b8dc3cb89aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.874709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e68cc-a9ef-4d1b-855b-b8dc3cb89aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.876191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94e68cc-a9ef-4d1b-855b-b8dc3cb89aff None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.889199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae8a3f-7d95-4ad0-a7bb-2dfb7ad6d313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:56.889854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8a3f-7d95-4ad0-a7bb-2dfb7ad6d313 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:56.890141 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8a3f-7d95-4ad0-a7bb-2dfb7ad6d313 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:56.890423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae8a3f-7d95-4ad0-a7bb-2dfb7ad6d313 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:56.891179 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae8a3f-7d95-4ad0-a7bb-2dfb7ad6d313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:56.891490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae8a3f-7d95-4ad0-a7bb-2dfb7ad6d313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:56.892861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae8a3f-7d95-4ad0-a7bb-2dfb7ad6d313 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f94e68cc-a9ef-4d1b-855b-b8dc3cb89aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:56.893991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:56.895630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:56.896763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:56.896763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:56.896763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:56.896763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:56.896763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 319/652] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:56.896763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e68cc-a9ef-4d1b-855b-b8dc3cb89aff None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.908890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 334/653] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:56.909715 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae8a3f-7d95-4ad0-a7bb-2dfb7ad6d313 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:56.948433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:56.956265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:56.999205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e68cc-a9ef-4d1b-855b-b8dc3cb89aff None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.010742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae8a3f-7d95-4ad0-a7bb-2dfb7ad6d313 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.032374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623df3a-e487-4e32-9f63-cd48a0629fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:57.033259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623df3a-e487-4e32-9f63-cd48a0629fef None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:57.033533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623df3a-e487-4e32-9f63-cd48a0629fef None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:57.033805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7623df3a-e487-4e32-9f63-cd48a0629fef None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/4c73aa1f244d4251b8539c2806f38a42/roles/67e4864efce9469bb6bdcca8ee5d5b24` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:57.044629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d4ee1-76a3-4d25-9c66-bb4cd3dcac12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:57.045281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d4ee1-76a3-4d25-9c66-bb4cd3dcac12 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:57.045566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d4ee1-76a3-4d25-9c66-bb4cd3dcac12 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:57.045848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d4ee1-76a3-4d25-9c66-bb4cd3dcac12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:57.046685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d4ee1-76a3-4d25-9c66-bb4cd3dcac12 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=e7a416f22d33450183801cef2b13822f) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:57.047000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d4ee1-76a3-4d25-9c66-bb4cd3dcac12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:57.047590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623df3a-e487-4e32-9f63-cd48a0629fef None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=4c73aa1f244d4251b8539c2806f38a42, role_id=67e4864efce9469bb6bdcca8ee5d5b24)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:57.048573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d4ee1-76a3-4d25-9c66-bb4cd3dcac12 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:57.049000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7623df3a-e487-4e32-9f63-cd48a0629fef None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:57.054391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 335/654] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:56 2022] GET /identity/v3/role_assignments?user.id=e7a416f22d33450183801cef2b13822f&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:57.055221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d4ee1-76a3-4d25-9c66-bb4cd3dcac12 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:57.061623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 320/655] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:00:56 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/4c73aa1f244d4251b8539c2806f38a42/roles/67e4864efce9469bb6bdcca8ee5d5b24 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:00:57.062477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623df3a-e487-4e32-9f63-cd48a0629fef None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:57.221122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:57.225919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d4ee1-76a3-4d25-9c66-bb4cd3dcac12 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.226479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7623df3a-e487-4e32-9f63-cd48a0629fef None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.241876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:57.273154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b23b1-d182-4f3a-9c30-616dcc462773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:57.273859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b23b1-d182-4f3a-9c30-616dcc462773 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:57.274393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b23b1-d182-4f3a-9c30-616dcc462773 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:57.274481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b23b1-d182-4f3a-9c30-616dcc462773 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:57.275349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b23b1-d182-4f3a-9c30-616dcc462773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:57.275469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44870682-890b-4613-9878-14fc3dd521c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:57.276301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b23b1-d182-4f3a-9c30-616dcc462773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:57.276406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44870682-890b-4613-9878-14fc3dd521c7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:57.276406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44870682-890b-4613-9878-14fc3dd521c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:57.276536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44870682-890b-4613-9878-14fc3dd521c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:57.277230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b23b1-d182-4f3a-9c30-616dcc462773 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:57.277332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44870682-890b-4613-9878-14fc3dd521c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:57.277638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44870682-890b-4613-9878-14fc3dd521c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:57.278981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44870682-890b-4613-9878-14fc3dd521c7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:57.290975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 336/656] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:57.291723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 321/657] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:57.291854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b23b1-d182-4f3a-9c30-616dcc462773 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:57.299336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44870682-890b-4613-9878-14fc3dd521c7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:57.411973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44870682-890b-4613-9878-14fc3dd521c7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.421959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b23b1-d182-4f3a-9c30-616dcc462773 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.458630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c20ac-e70c-4a53-920c-94bc4e5d5ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:57.459996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c20ac-e70c-4a53-920c-94bc4e5d5ccb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:57.460530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c20ac-e70c-4a53-920c-94bc4e5d5ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:57.461054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c20ac-e70c-4a53-920c-94bc4e5d5ccb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:57.461554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ecb18-60f4-437f-a912-e6e3f6955030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:57.462413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ecb18-60f4-437f-a912-e6e3f6955030 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:57.462766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ecb18-60f4-437f-a912-e6e3f6955030 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:57.463144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ecb18-60f4-437f-a912-e6e3f6955030 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:57.470830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ecb18-60f4-437f-a912-e6e3f6955030 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:57.471201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c20ac-e70c-4a53-920c-94bc4e5d5ccb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:57.472488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ecb18-60f4-437f-a912-e6e3f6955030 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:57.473057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c20ac-e70c-4a53-920c-94bc4e5d5ccb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-089ecb18-60f4-437f-a912-e6e3f6955030 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:57.486760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:57.487917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:57.489039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:57.489039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:57.489039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:57.489039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:57.489039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:57.489039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:57.489039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:57.489039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:57.490344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 337/658] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-1d6c20ac-e70c-4a53-920c-94bc4e5d5ccb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:57.491640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:57.492561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:57.493515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:57.493515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:57.493515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:57.493515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:57.493515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:57.493515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:57.493515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:57.493515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:57.493515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:00:57.493515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:57.495459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ecb18-60f4-437f-a912-e6e3f6955030 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:57.496402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 322/659] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:57.497633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c20ac-e70c-4a53-920c-94bc4e5d5ccb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:57.522770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:57.530467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:57.616719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ecb18-60f4-437f-a912-e6e3f6955030 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.635754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c20ac-e70c-4a53-920c-94bc4e5d5ccb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.653332 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1b5ae-a9fa-46ca-bba2-464cad0bc2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:57.654133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1b5ae-a9fa-46ca-bba2-464cad0bc2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:57.654416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1b5ae-a9fa-46ca-bba2-464cad0bc2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:57.654694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d1b5ae-a9fa-46ca-bba2-464cad0bc2a6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:57.663452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1b5ae-a9fa-46ca-bba2-464cad0bc2a6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:57.664910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d1b5ae-a9fa-46ca-bba2-464cad0bc2a6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-70d1b5ae-a9fa-46ca-bba2-464cad0bc2a6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:00:57.679421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:57.680402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:57.681357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:57.681357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:57.681357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:00:57.681357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:00:57.681357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:00:57.681357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:00:57.681357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:00:57.681357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:00:57.681357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 05 15:00:57.681357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:57.682639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a156834-7f22-4775-940c-ef72f0a87cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:57.684080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a156834-7f22-4775-940c-ef72f0a87cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:57.684509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a156834-7f22-4775-940c-ef72f0a87cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:57.684945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a156834-7f22-4775-940c-ef72f0a87cb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:57.685706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 338/660] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:57.686038 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a156834-7f22-4775-940c-ef72f0a87cb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:57.686497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a156834-7f22-4775-940c-ef72f0a87cb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:57.686608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1b5ae-a9fa-46ca-bba2-464cad0bc2a6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:57.688410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a156834-7f22-4775-940c-ef72f0a87cb7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:57.706313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 323/661] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:57.707787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a156834-7f22-4775-940c-ef72f0a87cb7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:57.798532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d1b5ae-a9fa-46ca-bba2-464cad0bc2a6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.803337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:57.819428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:57.829340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a156834-7f22-4775-940c-ef72f0a87cb7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.834659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4562de-efe4-489d-a545-9795e216a3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:57.835439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4562de-efe4-489d-a545-9795e216a3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:57.835730 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4562de-efe4-489d-a545-9795e216a3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:57.836012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4562de-efe4-489d-a545-9795e216a3a6 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:57.836866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4562de-efe4-489d-a545-9795e216a3a6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:57.837181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4562de-efe4-489d-a545-9795e216a3a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:57.838633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4562de-efe4-489d-a545-9795e216a3a6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:57.862716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 339/662] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 5 15:00:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:57.863765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4562de-efe4-489d-a545-9795e216a3a6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:57.868586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771841fa-08df-48c5-8919-7eb1ba581c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:57.869578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771841fa-08df-48c5-8919-7eb1ba581c9b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:57.870139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771841fa-08df-48c5-8919-7eb1ba581c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:57.870651 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771841fa-08df-48c5-8919-7eb1ba581c9b None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:57.872273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771841fa-08df-48c5-8919-7eb1ba581c9b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:57.872821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771841fa-08df-48c5-8919-7eb1ba581c9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:57.874494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771841fa-08df-48c5-8919-7eb1ba581c9b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:57.893052 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 324/663] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Jul 5 15:00:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:57.894224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771841fa-08df-48c5-8919-7eb1ba581c9b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:57.974419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4562de-efe4-489d-a545-9795e216a3a6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:57.997912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771841fa-08df-48c5-8919-7eb1ba581c9b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:58.011983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5852a-f89b-456e-a84a-ae84b0328f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:58.012645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5852a-f89b-456e-a84a-ae84b0328f97 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:58.012926 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5852a-f89b-456e-a84a-ae84b0328f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:58.013200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c5852a-f89b-456e-a84a-ae84b0328f97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:58.014192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5852a-f89b-456e-a84a-ae84b0328f97 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=450bcd0f22b840c299d08ce9433d6c85) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:58.014492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5852a-f89b-456e-a84a-ae84b0328f97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:58.016227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c5852a-f89b-456e-a84a-ae84b0328f97 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:58.023037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 340/664] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:57 2022] GET /identity/v3/role_assignments?user.id=450bcd0f22b840c299d08ce9433d6c85&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:58.023936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5852a-f89b-456e-a84a-ae84b0328f97 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:58.037519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:58.039072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e844abb-69fc-47ab-af90-e3d900640887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:58.040149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e844abb-69fc-47ab-af90-e3d900640887 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:58.040684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e844abb-69fc-47ab-af90-e3d900640887 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:58.041255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e844abb-69fc-47ab-af90-e3d900640887 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:58.047109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:58.151170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c5852a-f89b-456e-a84a-ae84b0328f97 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:58.196918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c227a3a-6d86-4322-b98d-c1fc4aa1d9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:58.197772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c227a3a-6d86-4322-b98d-c1fc4aa1d9ba None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:58.198947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c227a3a-6d86-4322-b98d-c1fc4aa1d9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:58.199461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c227a3a-6d86-4322-b98d-c1fc4aa1d9ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:58.216270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e844abb-69fc-47ab-af90-e3d900640887 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:58.218059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e844abb-69fc-47ab-af90-e3d900640887 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:58.284218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c227a3a-6d86-4322-b98d-c1fc4aa1d9ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:58.286320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c227a3a-6d86-4322-b98d-c1fc4aa1d9ba None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:58.300699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:58.343083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 325/665] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:58.348229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e844abb-69fc-47ab-af90-e3d900640887 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:58.386059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:58.412255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 341/666] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:00:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:00:58.417124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c227a3a-6d86-4322-b98d-c1fc4aa1d9ba None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:58.470560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e844abb-69fc-47ab-af90-e3d900640887 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:58.506756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684522ee-945d-4c34-b190-2a8b8178ff9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:58.507584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684522ee-945d-4c34-b190-2a8b8178ff9b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:58.507876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684522ee-945d-4c34-b190-2a8b8178ff9b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:58.508152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684522ee-945d-4c34-b190-2a8b8178ff9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:58.520080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684522ee-945d-4c34-b190-2a8b8178ff9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:58.521531 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684522ee-945d-4c34-b190-2a8b8178ff9b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-684522ee-945d-4c34-b190-2a8b8178ff9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:58.540387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:58.541412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:58.542368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:58.542368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:58.542368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:58.542368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:58.542368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 326/667] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:58.546778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c227a3a-6d86-4322-b98d-c1fc4aa1d9ba None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:58.552081 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684522ee-945d-4c34-b190-2a8b8178ff9b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:58.563067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:58.594091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3f2fa-53fb-4d06-b73a-d3023de32fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:58.595035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3f2fa-53fb-4d06-b73a-d3023de32fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:58.595443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3f2fa-53fb-4d06-b73a-d3023de32fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:58.595795 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3f2fa-53fb-4d06-b73a-d3023de32fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:58.607533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3f2fa-53fb-4d06-b73a-d3023de32fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:58.609108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3f2fa-53fb-4d06-b73a-d3023de32fe3 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-30e3f2fa-53fb-4d06-b73a-d3023de32fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:00:58.627786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:58.628719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:58.629617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:00:58.629617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:00:58.629617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:00:58.629617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:58.629617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 342/668] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:00:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:58.640121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3f2fa-53fb-4d06-b73a-d3023de32fe3 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:58.661247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:58.675672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684522ee-945d-4c34-b190-2a8b8178ff9b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:58.734999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e5502-cb5f-4f8b-84c9-ffda13e09014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:58.736094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e5502-cb5f-4f8b-84c9-ffda13e09014 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:58.736570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e5502-cb5f-4f8b-84c9-ffda13e09014 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:58.737090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e5502-cb5f-4f8b-84c9-ffda13e09014 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:58.738009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e5502-cb5f-4f8b-84c9-ffda13e09014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:58.738508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e5502-cb5f-4f8b-84c9-ffda13e09014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:58.740133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e5502-cb5f-4f8b-84c9-ffda13e09014 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:58.765535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 327/669] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:00:58.770718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e5502-cb5f-4f8b-84c9-ffda13e09014 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:58.814890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3f2fa-53fb-4d06-b73a-d3023de32fe3 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:58.878774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de49e11-0cbe-4110-8c66-f0df66296159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:58.879895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de49e11-0cbe-4110-8c66-f0df66296159 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:58.880380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de49e11-0cbe-4110-8c66-f0df66296159 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:58.880907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de49e11-0cbe-4110-8c66-f0df66296159 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:58.881870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de49e11-0cbe-4110-8c66-f0df66296159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:58.882474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de49e11-0cbe-4110-8c66-f0df66296159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:58.884173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de49e11-0cbe-4110-8c66-f0df66296159 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:58.895067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e5502-cb5f-4f8b-84c9-ffda13e09014 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:58.904585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 343/670] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:58.911122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de49e11-0cbe-4110-8c66-f0df66296159 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:58.933748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a01c45-5212-42a1-b570-892d6359fb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:58.934293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a01c45-5212-42a1-b570-892d6359fb02 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:58.934580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a01c45-5212-42a1-b570-892d6359fb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:58.934858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a01c45-5212-42a1-b570-892d6359fb02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:58.944790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a01c45-5212-42a1-b570-892d6359fb02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:58.944790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a01c45-5212-42a1-b570-892d6359fb02 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:58.953702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-88a01c45-5212-42a1-b570-892d6359fb02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:58.961063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:58.962216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:58.963367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:58.963367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:58.963367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:58.963367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:58.963367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:58.963367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:58.963367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:58.963367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:58.963367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 328/671] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:58.965623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a01c45-5212-42a1-b570-892d6359fb02 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:58.976895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:59.070064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a01c45-5212-42a1-b570-892d6359fb02 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:59.105805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b442e18-8b01-40f2-9d08-5ab207edac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:59.106655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b442e18-8b01-40f2-9d08-5ab207edac50 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.107055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b442e18-8b01-40f2-9d08-5ab207edac50 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.107441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b442e18-8b01-40f2-9d08-5ab207edac50 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.108348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b442e18-8b01-40f2-9d08-5ab207edac50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:59.108765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b442e18-8b01-40f2-9d08-5ab207edac50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:59.110304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b442e18-8b01-40f2-9d08-5ab207edac50 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:59.111287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de49e11-0cbe-4110-8c66-f0df66296159 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:59.124116 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 329/672] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:59.129549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b442e18-8b01-40f2-9d08-5ab207edac50 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:59.168240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e674e-6df3-4bcd-8207-eff1e1393c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:59.169156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e674e-6df3-4bcd-8207-eff1e1393c86 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.169446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e674e-6df3-4bcd-8207-eff1e1393c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.169728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e674e-6df3-4bcd-8207-eff1e1393c86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.177403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e674e-6df3-4bcd-8207-eff1e1393c86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:59.178854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e674e-6df3-4bcd-8207-eff1e1393c86 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-a05e674e-6df3-4bcd-8207-eff1e1393c86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:00:59.195478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:59.196407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:59.197822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:00:59.197822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:00:59.197822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:00:59.197822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:00:59.197822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:00:59.197822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:00:59.197822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:00:59.197822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:59.197822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 344/673] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:59.203506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e674e-6df3-4bcd-8207-eff1e1393c86 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:59.292281 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:59.296221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:59.318860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e674e-6df3-4bcd-8207-eff1e1393c86 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:59.328494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b442e18-8b01-40f2-9d08-5ab207edac50 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:59.360554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ed498-8a99-4c87-8686-31697f948556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:59.361734 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ed498-8a99-4c87-8686-31697f948556 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.362150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ed498-8a99-4c87-8686-31697f948556 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.362429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ed498-8a99-4c87-8686-31697f948556 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.363271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ed498-8a99-4c87-8686-31697f948556 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:59.363586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ed498-8a99-4c87-8686-31697f948556 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:59.365032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ed498-8a99-4c87-8686-31697f948556 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:59.365573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc6f7-23d0-479b-b0b4-2e48b65db7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:59.366230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc6f7-23d0-479b-b0b4-2e48b65db7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.366778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc6f7-23d0-479b-b0b4-2e48b65db7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.367080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc6f7-23d0-479b-b0b4-2e48b65db7a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.368388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc6f7-23d0-479b-b0b4-2e48b65db7a9 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=8ad7cee9ffed415b9a30176b1888e562) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:59.368703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc6f7-23d0-479b-b0b4-2e48b65db7a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:59.370119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc6f7-23d0-479b-b0b4-2e48b65db7a9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:59.378575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 330/674] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:59 2022] GET /identity/v3/role_assignments?user.id=8ad7cee9ffed415b9a30176b1888e562&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:59.379123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 345/675] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:00:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:59.384509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ed498-8a99-4c87-8686-31697f948556 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:59.389533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c336883-cf67-423b-9b2b-e563c325b22a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.389823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c336883-cf67-423b-9b2b-e563c325b22a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.390099 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c336883-cf67-423b-9b2b-e563c325b22a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.391828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 331/676] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:59 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:59.397602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949ed4-98e3-450b-b554-5f7a0696e91f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.397898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949ed4-98e3-450b-b554-5f7a0696e91f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.398158 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949ed4-98e3-450b-b554-5f7a0696e91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.432297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d2949ed4-98e3-450b-b554-5f7a0696e91f None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:59.434092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d2949ed4-98e3-450b-b554-5f7a0696e91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:59.528750 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ed498-8a99-4c87-8686-31697f948556 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:59.575468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd345d3-5711-4f4c-835f-b4399d42e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:59.577094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd345d3-5711-4f4c-835f-b4399d42e3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.577094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd345d3-5711-4f4c-835f-b4399d42e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.577284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd345d3-5711-4f4c-835f-b4399d42e3b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.578136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd345d3-5711-4f4c-835f-b4399d42e3b2 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=cd7fa93466c54632ad95834297410e00) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:00:59.578453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd345d3-5711-4f4c-835f-b4399d42e3b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:00:59.579996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd345d3-5711-4f4c-835f-b4399d42e3b2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:00:59.593128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 346/677] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:00:59 2022] GET /identity/v3/role_assignments?user.id=cd7fa93466c54632ad95834297410e00&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:00:59.594427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1edcf3-decd-4834-95f2-4ccf5823f894 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.594716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1edcf3-decd-4834-95f2-4ccf5823f894 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.595129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1edcf3-decd-4834-95f2-4ccf5823f894 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.596652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 347/678] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:00:59.601827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e29927-d574-41ad-8856-06e2148cc3e6 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.602112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e29927-d574-41ad-8856-06e2148cc3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.602387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e29927-d574-41ad-8856-06e2148cc3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.605946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 332/679] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 05 15:00:59.616087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:59.621550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-50e29927-d574-41ad-8856-06e2148cc3e6 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:00:59.622777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-50e29927-d574-41ad-8856-06e2148cc3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:00:59.636558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2949ed4-98e3-450b-b554-5f7a0696e91f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:59.727159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 348/680] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 05 15:00:59.738372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:59.763252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e29927-d574-41ad-8856-06e2148cc3e6 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:59.781513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2949ed4-98e3-450b-b554-5f7a0696e91f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:59.817660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7eef4-b84e-4e44-9450-055c0110e9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:59.819027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7eef4-b84e-4e44-9450-055c0110e9da None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.819624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7eef4-b84e-4e44-9450-055c0110e9da None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.820126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7eef4-b84e-4e44-9450-055c0110e9da None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.831618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7eef4-b84e-4e44-9450-055c0110e9da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-12c7eef4-b84e-4e44-9450-055c0110e9da None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:59.840328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:59.841273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:00:59.842120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:00:59.844867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 333/681] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:00:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:00:59.852929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7eef4-b84e-4e44-9450-055c0110e9da None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:59.879959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e29927-d574-41ad-8856-06e2148cc3e6 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:00:59.918349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4d08a-5c1c-405f-80ce-a57939c3ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:00:59.919178 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4d08a-5c1c-405f-80ce-a57939c3ef35 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:00:59.919468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4d08a-5c1c-405f-80ce-a57939c3ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:00:59.919751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4d08a-5c1c-405f-80ce-a57939c3ef35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:00:59.922591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:00:59.932423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4d08a-5c1c-405f-80ce-a57939c3ef35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-7ab4d08a-5c1c-405f-80ce-a57939c3ef35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:00:59.940674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:00:59.941682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:00:59.942530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 349/682] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:00:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:00:59.947580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4d08a-5c1c-405f-80ce-a57939c3ef35 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:00:59.993067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7eef4-b84e-4e44-9450-055c0110e9da None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:00.021011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:00.036801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f7503-b5dc-4c64-834c-fa7e09ff4a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:00.037402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f7503-b5dc-4c64-834c-fa7e09ff4a80 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:00.037683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f7503-b5dc-4c64-834c-fa7e09ff4a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:00.037957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f7503-b5dc-4c64-834c-fa7e09ff4a80 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:00.038704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f7503-b5dc-4c64-834c-fa7e09ff4a80 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:00.039012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f7503-b5dc-4c64-834c-fa7e09ff4a80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:00.040414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f7503-b5dc-4c64-834c-fa7e09ff4a80 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:00.052752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 334/683] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:00:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:00.058693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f7503-b5dc-4c64-834c-fa7e09ff4a80 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:00.087621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4d08a-5c1c-405f-80ce-a57939c3ef35 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:00.143424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08804-7e6e-4e47-a758-eb3e1c5ec32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:00.144085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08804-7e6e-4e47-a758-eb3e1c5ec32a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:00.144369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08804-7e6e-4e47-a758-eb3e1c5ec32a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:00.144643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08804-7e6e-4e47-a758-eb3e1c5ec32a None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:00.145402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e08804-7e6e-4e47-a758-eb3e1c5ec32a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:00.145710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e08804-7e6e-4e47-a758-eb3e1c5ec32a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:00.149637 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e08804-7e6e-4e47-a758-eb3e1c5ec32a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:00.168826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 350/684] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:00:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:00.173935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08804-7e6e-4e47-a758-eb3e1c5ec32a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:00.233371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f7503-b5dc-4c64-834c-fa7e09ff4a80 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:00.285238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:00.297122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58ff2c-7218-4251-bc6e-bf591c8f2839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:00.297844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58ff2c-7218-4251-bc6e-bf591c8f2839 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:00.298126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58ff2c-7218-4251-bc6e-bf591c8f2839 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:00.298399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa58ff2c-7218-4251-bc6e-bf591c8f2839 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:00.318034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58ff2c-7218-4251-bc6e-bf591c8f2839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:00.321458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa58ff2c-7218-4251-bc6e-bf591c8f2839 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-aa58ff2c-7218-4251-bc6e-bf591c8f2839 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:00.347008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:00.347979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:00.348876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:00.348876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:00.348876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:00.348876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:00.357472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 335/685] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:00.363381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58ff2c-7218-4251-bc6e-bf591c8f2839 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:00.365042 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08804-7e6e-4e47-a758-eb3e1c5ec32a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:00.408999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50a1c-4b9d-40ca-8ef8-2c0faa159ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:00.409856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50a1c-4b9d-40ca-8ef8-2c0faa159ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:00.410149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50a1c-4b9d-40ca-8ef8-2c0faa159ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:00.410431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf50a1c-4b9d-40ca-8ef8-2c0faa159ba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:00.433146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:00.437428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf50a1c-4b9d-40ca-8ef8-2c0faa159ba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:00.438908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf50a1c-4b9d-40ca-8ef8-2c0faa159ba4 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-bcf50a1c-4b9d-40ca-8ef8-2c0faa159ba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:00.469622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:00.471693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:00.473805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:00.473805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:00.473805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:00.473805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:00.475391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 351/686] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:00.480403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50a1c-4b9d-40ca-8ef8-2c0faa159ba4 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:00.578060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa58ff2c-7218-4251-bc6e-bf591c8f2839 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:00.583820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf50a1c-4b9d-40ca-8ef8-2c0faa159ba4 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:00.595235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:00.625784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f939ed6-6eba-4082-9199-c278e9306897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:00.626453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f939ed6-6eba-4082-9199-c278e9306897 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:00.626742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f939ed6-6eba-4082-9199-c278e9306897 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:00.627022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f939ed6-6eba-4082-9199-c278e9306897 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:00.627789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f939ed6-6eba-4082-9199-c278e9306897 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:00.628100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f939ed6-6eba-4082-9199-c278e9306897 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:00.629475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f939ed6-6eba-4082-9199-c278e9306897 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:00.647711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97efee9-17cb-4cf1-8f97-538d989f2945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:00.648382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97efee9-17cb-4cf1-8f97-538d989f2945 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:00.648668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97efee9-17cb-4cf1-8f97-538d989f2945 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:00.648946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97efee9-17cb-4cf1-8f97-538d989f2945 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:00.649686 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97efee9-17cb-4cf1-8f97-538d989f2945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:00.649997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97efee9-17cb-4cf1-8f97-538d989f2945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:00.651391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97efee9-17cb-4cf1-8f97-538d989f2945 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:00.655355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 352/687] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:00.660841 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f939ed6-6eba-4082-9199-c278e9306897 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:00.680743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 336/688] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:00.687159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97efee9-17cb-4cf1-8f97-538d989f2945 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:00.812947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:00.819245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97efee9-17cb-4cf1-8f97-538d989f2945 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:00.842573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f939ed6-6eba-4082-9199-c278e9306897 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:00.855021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c8efa-9a06-4cb6-a7c8-f43b2052a766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:00.856168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c8efa-9a06-4cb6-a7c8-f43b2052a766 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:00.856740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c8efa-9a06-4cb6-a7c8-f43b2052a766 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:00.857029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c8efa-9a06-4cb6-a7c8-f43b2052a766 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:00.866989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c8efa-9a06-4cb6-a7c8-f43b2052a766 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:00.868791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c8efa-9a06-4cb6-a7c8-f43b2052a766 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:00.881774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab9e45-1467-4fa1-82a0-473c599adc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:00.882592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab9e45-1467-4fa1-82a0-473c599adc90 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:00.882873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab9e45-1467-4fa1-82a0-473c599adc90 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:00.883175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab9e45-1467-4fa1-82a0-473c599adc90 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-5e0c8efa-9a06-4cb6-a7c8-f43b2052a766 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:00.888216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:00.889161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:00.890725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 337/689] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:00.891403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab9e45-1467-4fa1-82a0-473c599adc90 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:00.891483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbc083-3756-448c-ae13-7b590aad2da3 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:00.891741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbc083-3756-448c-ae13-7b590aad2da3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:00.891886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfbc083-3756-448c-ae13-7b590aad2da3 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:00.893398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab9e45-1467-4fa1-82a0-473c599adc90 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:00.893566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 338/690] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:00.898852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbc083-3756-448c-ae13-7b590aad2da3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-efab9e45-1467-4fa1-82a0-473c599adc90 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:00.909450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:00.910418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:00.911389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 353/691] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 5 15:01:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:00.912880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53d1f-b122-4ff7-bf14-da8e04a36ab7 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:00.912880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53d1f-b122-4ff7-bf14-da8e04a36ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:00.912880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba53d1f-b122-4ff7-bf14-da8e04a36ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:00.929965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4ba53d1f-b122-4ff7-bf14-da8e04a36ab7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:00.931201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4ba53d1f-b122-4ff7-bf14-da8e04a36ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:00.956325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:01.007174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 354/692] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:01:01.007174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba53d1f-b122-4ff7-bf14-da8e04a36ab7 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:01.018467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfbc083-3756-448c-ae13-7b590aad2da3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:01.069398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a227809-5ee3-4229-b6cd-f955ca76f29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:01.070914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a227809-5ee3-4229-b6cd-f955ca76f29c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:01.071457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a227809-5ee3-4229-b6cd-f955ca76f29c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:01.072175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a227809-5ee3-4229-b6cd-f955ca76f29c None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:01.073370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a227809-5ee3-4229-b6cd-f955ca76f29c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:01.074359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a227809-5ee3-4229-b6cd-f955ca76f29c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:01.076789 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a227809-5ee3-4229-b6cd-f955ca76f29c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:01.096451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 339/693] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 5 15:01:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:01.097219 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a227809-5ee3-4229-b6cd-f955ca76f29c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:01.111346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba53d1f-b122-4ff7-bf14-da8e04a36ab7 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:01.114036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:01.161445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320f29f-f308-496a-ab9f-8d7588cb71b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:01.162532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320f29f-f308-496a-ab9f-8d7588cb71b4 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:01.163108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320f29f-f308-496a-ab9f-8d7588cb71b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:01.163404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9320f29f-f308-496a-ab9f-8d7588cb71b4 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:01.164494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320f29f-f308-496a-ab9f-8d7588cb71b4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:01.165214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320f29f-f308-496a-ab9f-8d7588cb71b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:01.167188 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9320f29f-f308-496a-ab9f-8d7588cb71b4 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:01.194937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 355/694] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 5 15:01:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:01.195895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320f29f-f308-496a-ab9f-8d7588cb71b4 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:01.217217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a227809-5ee3-4229-b6cd-f955ca76f29c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:01.222151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:01.257311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3fdf4-4035-4402-910f-7fc9a1e0169c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:01.258393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3fdf4-4035-4402-910f-7fc9a1e0169c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:01.258878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3fdf4-4035-4402-910f-7fc9a1e0169c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:01.259338 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3fdf4-4035-4402-910f-7fc9a1e0169c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:01.270803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3fdf4-4035-4402-910f-7fc9a1e0169c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:01.272592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3fdf4-4035-4402-910f-7fc9a1e0169c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-79e3fdf4-4035-4402-910f-7fc9a1e0169c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:01.292042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:01.293708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:01.294618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:01.294618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:01.294618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:01.294618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:01.296547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 340/695] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:01.297685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3fdf4-4035-4402-910f-7fc9a1e0169c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:01.315240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9320f29f-f308-496a-ab9f-8d7588cb71b4 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:01.361164 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296379f8-c81a-4a2e-bcfe-92e498dbea0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:01.362044 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296379f8-c81a-4a2e-bcfe-92e498dbea0e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:01.362337 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296379f8-c81a-4a2e-bcfe-92e498dbea0e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:01.362618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296379f8-c81a-4a2e-bcfe-92e498dbea0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:01.369075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:01.408670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3fdf4-4035-4402-910f-7fc9a1e0169c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:01.445119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296379f8-c81a-4a2e-bcfe-92e498dbea0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:01.445876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0726f57-3684-4fcb-8be6-e2dc0395e9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:01.446574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296379f8-c81a-4a2e-bcfe-92e498dbea0e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:01.447031 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0726f57-3684-4fcb-8be6-e2dc0395e9ad None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:01.447550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0726f57-3684-4fcb-8be6-e2dc0395e9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:01.448028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0726f57-3684-4fcb-8be6-e2dc0395e9ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:01.468314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:01.524517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0726f57-3684-4fcb-8be6-e2dc0395e9ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:01.526583 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0726f57-3684-4fcb-8be6-e2dc0395e9ad None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:01.568714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 356/696] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:01:01.569788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296379f8-c81a-4a2e-bcfe-92e498dbea0e None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:01.639705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:01.691152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 341/697] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:01:01.693037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0726f57-3684-4fcb-8be6-e2dc0395e9ad None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:01.705853 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296379f8-c81a-4a2e-bcfe-92e498dbea0e None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:01.738858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:01.770994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fea73-0826-4d63-ab87-63583d6b750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:01.772834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fea73-0826-4d63-ab87-63583d6b750e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:01.773759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fea73-0826-4d63-ab87-63583d6b750e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:01.774574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fea73-0826-4d63-ab87-63583d6b750e None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:01.776311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fea73-0826-4d63-ab87-63583d6b750e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:01.777305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fea73-0826-4d63-ab87-63583d6b750e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:01.780334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fea73-0826-4d63-ab87-63583d6b750e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:01.802953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 357/698] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:01.805101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fea73-0826-4d63-ab87-63583d6b750e None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:01.821562 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0726f57-3684-4fcb-8be6-e2dc0395e9ad None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:01.858263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88292ba6-1d5b-44f3-a546-1e45c0a7136a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:01.859103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88292ba6-1d5b-44f3-a546-1e45c0a7136a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:01.859389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88292ba6-1d5b-44f3-a546-1e45c0a7136a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:01.859665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88292ba6-1d5b-44f3-a546-1e45c0a7136a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:01.870077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88292ba6-1d5b-44f3-a546-1e45c0a7136a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:01.872096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88292ba6-1d5b-44f3-a546-1e45c0a7136a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-88292ba6-1d5b-44f3-a546-1e45c0a7136a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:01.890855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:01.891796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:01.892687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:01.892687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:01.892687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:01.892687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:01.892687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 342/699] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:01.892974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88292ba6-1d5b-44f3-a546-1e45c0a7136a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:01.913246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:01.919298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fea73-0826-4d63-ab87-63583d6b750e None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:01.955018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ad72a0-3063-44b8-a456-23c4d7490907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:01.955839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad72a0-3063-44b8-a456-23c4d7490907 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:01.956129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad72a0-3063-44b8-a456-23c4d7490907 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:01.956409 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ad72a0-3063-44b8-a456-23c4d7490907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:01.967119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad72a0-3063-44b8-a456-23c4d7490907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:01.968610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ad72a0-3063-44b8-a456-23c4d7490907 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-51ad72a0-3063-44b8-a456-23c4d7490907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:01.986362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:01.987325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:01.988277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:01.988277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:01.988277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:01.988277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:01.988277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 358/700] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:01.988789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab349e-2bef-4b81-8011-4eaa11e0c13a None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:01.989082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab349e-2bef-4b81-8011-4eaa11e0c13a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:01.989366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab349e-2bef-4b81-8011-4eaa11e0c13a None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:01.990910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 359/701] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:01.991658 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab349e-2bef-4b81-8011-4eaa11e0c13a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:02.008374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88292ba6-1d5b-44f3-a546-1e45c0a7136a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:02.023512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:02.045615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe457389-0bb2-45fd-9307-91f68f86a314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:02.046314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe457389-0bb2-45fd-9307-91f68f86a314 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.046603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe457389-0bb2-45fd-9307-91f68f86a314 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.046881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe457389-0bb2-45fd-9307-91f68f86a314 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.047618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe457389-0bb2-45fd-9307-91f68f86a314 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:02.049076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe457389-0bb2-45fd-9307-91f68f86a314 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:02.075450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 343/702] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:01:01 2022] POST /identity/v3/users => generated 254 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:02.076879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe457389-0bb2-45fd-9307-91f68f86a314 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:02.120171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab349e-2bef-4b81-8011-4eaa11e0c13a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:02.169837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ad91e-7378-4c4e-877f-7359d04bbe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:02.171235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ad91e-7378-4c4e-877f-7359d04bbe70 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.171346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:02.172157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ad91e-7378-4c4e-877f-7359d04bbe70 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.172436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ad91e-7378-4c4e-877f-7359d04bbe70 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.173297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ad91e-7378-4c4e-877f-7359d04bbe70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:02.173630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ad91e-7378-4c4e-877f-7359d04bbe70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:02.175059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ad91e-7378-4c4e-877f-7359d04bbe70 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:02.188241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 360/703] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:02.189690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a53767-8655-41e2-8fee-4da77f8c161c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.189984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a53767-8655-41e2-8fee-4da77f8c161c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.190263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a53767-8655-41e2-8fee-4da77f8c161c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.210503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe457389-0bb2-45fd-9307-91f68f86a314 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:02.215682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-07a53767-8655-41e2-8fee-4da77f8c161c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:02.216905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-07a53767-8655-41e2-8fee-4da77f8c161c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:02.251947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca601e5-8dab-49ba-8b19-e16739fae9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:02.252615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca601e5-8dab-49ba-8b19-e16739fae9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.252899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca601e5-8dab-49ba-8b19-e16739fae9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.253171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca601e5-8dab-49ba-8b19-e16739fae9a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.253894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca601e5-8dab-49ba-8b19-e16739fae9a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:02.254200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca601e5-8dab-49ba-8b19-e16739fae9a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:02.255805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca601e5-8dab-49ba-8b19-e16739fae9a0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:02.272224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 344/704] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:02.273571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb199ae-f435-44c1-9e7a-55069e5cab83 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.274160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb199ae-f435-44c1-9e7a-55069e5cab83 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.274160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb199ae-f435-44c1-9e7a-55069e5cab83 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.275676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 345/705] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:02.276378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb199ae-f435-44c1-9e7a-55069e5cab83 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:02.296421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 361/706] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:01:02.297712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2527c-09c6-48fc-8913-6979bc65a7c4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.298004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2527c-09c6-48fc-8913-6979bc65a7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.298288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b2527c-09c6-48fc-8913-6979bc65a7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.326528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-53b2527c-09c6-48fc-8913-6979bc65a7c4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:02.328084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-53b2527c-09c6-48fc-8913-6979bc65a7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:02.344240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:02.386961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb199ae-f435-44c1-9e7a-55069e5cab83 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:02.402379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 362/707] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:01:02.411997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2527c-09c6-48fc-8913-6979bc65a7c4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:02.422424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95810a-4e33-4aa9-bed0-1cca71edee4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:02.423188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95810a-4e33-4aa9-bed0-1cca71edee4b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.423553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95810a-4e33-4aa9-bed0-1cca71edee4b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.423831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf95810a-4e33-4aa9-bed0-1cca71edee4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.425842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:02.431228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95810a-4e33-4aa9-bed0-1cca71edee4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:02.433830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf95810a-4e33-4aa9-bed0-1cca71edee4b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-bf95810a-4e33-4aa9-bed0-1cca71edee4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:02.447459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:02.448433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:02.449409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:02.449409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:02.449409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:02.449409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:02.449409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:02.449409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:02.449409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:02.449409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:02.449409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 346/708] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:02.450586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95810a-4e33-4aa9-bed0-1cca71edee4b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:02.524082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b2527c-09c6-48fc-8913-6979bc65a7c4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:02.558570 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ae479-a32a-417c-8b3b-ad9bf696a7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:02.559663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ae479-a32a-417c-8b3b-ad9bf696a7cb None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.560139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ae479-a32a-417c-8b3b-ad9bf696a7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.560416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ae479-a32a-417c-8b3b-ad9bf696a7cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.565215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf95810a-4e33-4aa9-bed0-1cca71edee4b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:02.567315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ae479-a32a-417c-8b3b-ad9bf696a7cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:02.568826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ae479-a32a-417c-8b3b-ad9bf696a7cb None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-5d4ae479-a32a-417c-8b3b-ad9bf696a7cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:02.583694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:02.584640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:02.585580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:02.585580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:02.585580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:02.585580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:02.585580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:02.585580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:02.585580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:02.585580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:02.585580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 363/709] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:02.586067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ae479-a32a-417c-8b3b-ad9bf696a7cb None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:02.602730 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:02.609397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd5c94-b804-412a-9a56-a204285b7f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:02.610369 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd5c94-b804-412a-9a56-a204285b7f95 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.610935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd5c94-b804-412a-9a56-a204285b7f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.611257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd5c94-b804-412a-9a56-a204285b7f95 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.611919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd5c94-b804-412a-9a56-a204285b7f95 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:02.613714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd5c94-b804-412a-9a56-a204285b7f95 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8cdd5c94-b804-412a-9a56-a204285b7f95 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:02.631757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:02.632751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:02.638850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 347/710] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Jul 5 15:01:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:02.639761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd5c94-b804-412a-9a56-a204285b7f95 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:02.692264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ae479-a32a-417c-8b3b-ad9bf696a7cb None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:02.700738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:02.727003 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb11883-58b6-471e-8126-877ba59edfa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:02.727830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb11883-58b6-471e-8126-877ba59edfa2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.728115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb11883-58b6-471e-8126-877ba59edfa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.728391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb11883-58b6-471e-8126-877ba59edfa2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.729053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb11883-58b6-471e-8126-877ba59edfa2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:02.730473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb11883-58b6-471e-8126-877ba59edfa2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-edb11883-58b6-471e-8126-877ba59edfa2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:02.744344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:02.746131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:02.747180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 364/711] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:02 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:02.748228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb11883-58b6-471e-8126-877ba59edfa2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:02.751374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd5c94-b804-412a-9a56-a204285b7f95 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:02.788372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2a8cf-7c52-4d6d-97e4-045cd00b441c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:02.789145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a8cf-7c52-4d6d-97e4-045cd00b441c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.789499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a8cf-7c52-4d6d-97e4-045cd00b441c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.789844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c2a8cf-7c52-4d6d-97e4-045cd00b441c None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.790707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2a8cf-7c52-4d6d-97e4-045cd00b441c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:02.791111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2a8cf-7c52-4d6d-97e4-045cd00b441c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:02.793127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c2a8cf-7c52-4d6d-97e4-045cd00b441c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:02.811353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 348/712] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:02.811353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2a8cf-7c52-4d6d-97e4-045cd00b441c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:02.834196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:02.860637 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb11883-58b6-471e-8126-877ba59edfa2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:02.896060 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74c77-33ad-4835-9f25-e75fb652da06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:02.896724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74c77-33ad-4835-9f25-e75fb652da06 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.897015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74c77-33ad-4835-9f25-e75fb652da06 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.897297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e74c77-33ad-4835-9f25-e75fb652da06 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.898093 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e74c77-33ad-4835-9f25-e75fb652da06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:02.898408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e74c77-33ad-4835-9f25-e75fb652da06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:02.899894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e74c77-33ad-4835-9f25-e75fb652da06 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:02.917038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 365/713] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:02.917038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74c77-33ad-4835-9f25-e75fb652da06 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:02.923748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c2a8cf-7c52-4d6d-97e4-045cd00b441c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:02.942252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:02.960640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70f561-242a-4020-a6eb-a0a14ba12609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:02.961231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70f561-242a-4020-a6eb-a0a14ba12609 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:02.961512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70f561-242a-4020-a6eb-a0a14ba12609 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:02.961784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70f561-242a-4020-a6eb-a0a14ba12609 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:02.962485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70f561-242a-4020-a6eb-a0a14ba12609 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:02.962788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70f561-242a-4020-a6eb-a0a14ba12609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:02.964731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d70f561-242a-4020-a6eb-a0a14ba12609 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:02.969354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 349/714] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 5 15:01:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:02.970202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70f561-242a-4020-a6eb-a0a14ba12609 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:03.023170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e74c77-33ad-4835-9f25-e75fb652da06 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:03.057474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949d7d8-7e86-45d2-84aa-eb4fff55ab65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:03.058314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949d7d8-7e86-45d2-84aa-eb4fff55ab65 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:03.058670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949d7d8-7e86-45d2-84aa-eb4fff55ab65 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:03.059010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949d7d8-7e86-45d2-84aa-eb4fff55ab65 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:03.059757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949d7d8-7e86-45d2-84aa-eb4fff55ab65 None admin] RBAC: Adding query filter params (name=network) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:03.060069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949d7d8-7e86-45d2-84aa-eb4fff55ab65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:03.061437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949d7d8-7e86-45d2-84aa-eb4fff55ab65 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:03.062878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:03.071181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 366/715] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:03.071181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949d7d8-7e86-45d2-84aa-eb4fff55ab65 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:03.079002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70f561-242a-4020-a6eb-a0a14ba12609 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:03.112970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14b5b1-a286-478b-8dfa-f385b2a82bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:03.113626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14b5b1-a286-478b-8dfa-f385b2a82bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:03.113921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14b5b1-a286-478b-8dfa-f385b2a82bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:03.114202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14b5b1-a286-478b-8dfa-f385b2a82bc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:03.115058 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14b5b1-a286-478b-8dfa-f385b2a82bc7 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=4c73aa1f244d4251b8539c2806f38a42) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:03.115384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14b5b1-a286-478b-8dfa-f385b2a82bc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:03.116815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14b5b1-a286-478b-8dfa-f385b2a82bc7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:03.131078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 350/716] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:02 2022] GET /identity/v3/role_assignments?user.id=4c73aa1f244d4251b8539c2806f38a42&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:03.131078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14b5b1-a286-478b-8dfa-f385b2a82bc7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:03.184924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:03.191215 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949d7d8-7e86-45d2-84aa-eb4fff55ab65 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:03.233506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5548085-9941-4575-8366-86c1ebe5777c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:03.234282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5548085-9941-4575-8366-86c1ebe5777c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:03.234642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5548085-9941-4575-8366-86c1ebe5777c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:03.234992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5548085-9941-4575-8366-86c1ebe5777c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:03.235957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5548085-9941-4575-8366-86c1ebe5777c None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=e7a416f22d33450183801cef2b13822f) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:03.236347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5548085-9941-4575-8366-86c1ebe5777c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:03.238006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5548085-9941-4575-8366-86c1ebe5777c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:03.245043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 367/717] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:03 2022] GET /identity/v3/role_assignments?user.id=e7a416f22d33450183801cef2b13822f&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:03.246134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5548085-9941-4575-8366-86c1ebe5777c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:03.265345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14b5b1-a286-478b-8dfa-f385b2a82bc7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:03.309235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5813c57-9222-4518-b677-2cdded5111b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:03.309921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5813c57-9222-4518-b677-2cdded5111b0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:03.310678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5813c57-9222-4518-b677-2cdded5111b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:03.311016 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5813c57-9222-4518-b677-2cdded5111b0 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:03.316026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5813c57-9222-4518-b677-2cdded5111b0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:03.316351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5813c57-9222-4518-b677-2cdded5111b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:03.317779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5813c57-9222-4518-b677-2cdded5111b0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:03.327436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:03.332321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 351/718] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 5 15:01:03 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:03.395717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5548085-9941-4575-8366-86c1ebe5777c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:03.439911 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b9b913-d7b2-4463-86d6-1cc4c675bdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:03.440607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b913-d7b2-4463-86d6-1cc4c675bdcc None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:03.440894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b913-d7b2-4463-86d6-1cc4c675bdcc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:03.441170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b9b913-d7b2-4463-86d6-1cc4c675bdcc None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:03.441902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9b913-d7b2-4463-86d6-1cc4c675bdcc None admin] RBAC: Adding query filter params (type=network) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:03.442288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9b913-d7b2-4463-86d6-1cc4c675bdcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:03.443741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b9b913-d7b2-4463-86d6-1cc4c675bdcc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:03.450432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 368/719] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:04.226390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23f2e2-2d15-4b51-a180-de242c423adb None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:04.226921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23f2e2-2d15-4b51-a180-de242c423adb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:04.226921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23f2e2-2d15-4b51-a180-de242c423adb None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:04.228696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 352/720] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:04.234247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af04f1-8a28-4df1-a346-48d3e0e24644 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:04.235150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af04f1-8a28-4df1-a346-48d3e0e24644 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:04.235662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af04f1-8a28-4df1-a346-48d3e0e24644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:04.254964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c2af04f1-8a28-4df1-a346-48d3e0e24644 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:04.256548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c2af04f1-8a28-4df1-a346-48d3e0e24644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:04.341208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 369/721] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:01:04.376426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23f2e2-2d15-4b51-a180-de242c423adb None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:04.507876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23f2e2-2d15-4b51-a180-de242c423adb None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:04.528583 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:04.561062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7df241-be2d-4235-ab87-67fdc6c85256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:04.561860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7df241-be2d-4235-ab87-67fdc6c85256 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:04.562145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7df241-be2d-4235-ab87-67fdc6c85256 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:04.562424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7df241-be2d-4235-ab87-67fdc6c85256 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:04.577157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7df241-be2d-4235-ab87-67fdc6c85256 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-1d7df241-be2d-4235-ab87-67fdc6c85256 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:04.584766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:04.585709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:04.586600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 353/722] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:01:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:04.592626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af04f1-8a28-4df1-a346-48d3e0e24644 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:04.733362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:04.784166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af04f1-8a28-4df1-a346-48d3e0e24644 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:04.852734 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02852f-7c24-4beb-af15-e80906a35134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:04.853421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02852f-7c24-4beb-af15-e80906a35134 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:04.853708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02852f-7c24-4beb-af15-e80906a35134 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:04.853990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02852f-7c24-4beb-af15-e80906a35134 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:04.855296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02852f-7c24-4beb-af15-e80906a35134 None admin] RBAC: Adding query filter params (name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:04.855610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02852f-7c24-4beb-af15-e80906a35134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:04.857002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02852f-7c24-4beb-af15-e80906a35134 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:04.874650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 370/723] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:01:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:04.884190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7df241-be2d-4235-ab87-67fdc6c85256 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:05.035713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7df241-be2d-4235-ab87-67fdc6c85256 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:05.090135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feed7c1-ddfc-4e3d-9797-4fcdd96ddbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:05.090944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feed7c1-ddfc-4e3d-9797-4fcdd96ddbbb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:05.091263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feed7c1-ddfc-4e3d-9797-4fcdd96ddbbb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:05.091541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feed7c1-ddfc-4e3d-9797-4fcdd96ddbbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:05.104630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feed7c1-ddfc-4e3d-9797-4fcdd96ddbbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:05.106139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feed7c1-ddfc-4e3d-9797-4fcdd96ddbbb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-1feed7c1-ddfc-4e3d-9797-4fcdd96ddbbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:05.128293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:05.129259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:05.130231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:05.130231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:05.130231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:05.130231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:05.130231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 354/724] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:05.134977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02852f-7c24-4beb-af15-e80906a35134 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:05.265492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02852f-7c24-4beb-af15-e80906a35134 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:05.282588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:05.310231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305fdaa-b710-4b09-b514-f90c48a8be59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:05.310825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305fdaa-b710-4b09-b514-f90c48a8be59 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:05.311131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305fdaa-b710-4b09-b514-f90c48a8be59 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:05.311412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7305fdaa-b710-4b09-b514-f90c48a8be59 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:05.312142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305fdaa-b710-4b09-b514-f90c48a8be59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:05.312453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305fdaa-b710-4b09-b514-f90c48a8be59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:05.313822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7305fdaa-b710-4b09-b514-f90c48a8be59 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:05.326785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 371/725] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:05.332357 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feed7c1-ddfc-4e3d-9797-4fcdd96ddbbb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:05.381877 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:05.489274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feed7c1-ddfc-4e3d-9797-4fcdd96ddbbb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:05.541124 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e15ba4-2d36-4bfb-b8a0-1b9ae11682eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:05.541948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e15ba4-2d36-4bfb-b8a0-1b9ae11682eb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:05.542765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e15ba4-2d36-4bfb-b8a0-1b9ae11682eb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:05.543082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e15ba4-2d36-4bfb-b8a0-1b9ae11682eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:05.557334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e15ba4-2d36-4bfb-b8a0-1b9ae11682eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:05.558926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e15ba4-2d36-4bfb-b8a0-1b9ae11682eb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f3e15ba4-2d36-4bfb-b8a0-1b9ae11682eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:05.578328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:05.579301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:05.581078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 355/726] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 5 15:01:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:05.585038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305fdaa-b710-4b09-b514-f90c48a8be59 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:05.750696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7305fdaa-b710-4b09-b514-f90c48a8be59 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:05.797656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb70425-44d5-4a9b-9a09-8e1494c09671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:05.798339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb70425-44d5-4a9b-9a09-8e1494c09671 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:05.798627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb70425-44d5-4a9b-9a09-8e1494c09671 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:05.798906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb70425-44d5-4a9b-9a09-8e1494c09671 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:05.799787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb70425-44d5-4a9b-9a09-8e1494c09671 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:05.800108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb70425-44d5-4a9b-9a09-8e1494c09671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:05.801555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb70425-44d5-4a9b-9a09-8e1494c09671 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:05.825286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 372/727] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 5 15:01:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:05.831098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e15ba4-2d36-4bfb-b8a0-1b9ae11682eb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:05.832679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988fdaf-f5cd-4085-b2a7-1b94e6719608 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:05.832978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988fdaf-f5cd-4085-b2a7-1b94e6719608 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:05.833264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b988fdaf-f5cd-4085-b2a7-1b94e6719608 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:05.834843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 373/728] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:05.848651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf69366-1cc3-4686-9045-d4c643bfd35c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:05.848949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf69366-1cc3-4686-9045-d4c643bfd35c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:05.849221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf69366-1cc3-4686-9045-d4c643bfd35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:05.871239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-bbf69366-1cc3-4686-9045-d4c643bfd35c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:05.872472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bbf69366-1cc3-4686-9045-d4c643bfd35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:05.904015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:05.938864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:05.954478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 374/729] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:01:05.955825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2d35d-6500-42d1-bc73-7d008dec93b1 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:05.956117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2d35d-6500-42d1-bc73-7d008dec93b1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:05.956401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2d35d-6500-42d1-bc73-7d008dec93b1 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:05.961634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 375/730] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:05.961634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c4622-1830-4c3a-8598-fd8de2cb3dcc None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:05.961634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c4622-1830-4c3a-8598-fd8de2cb3dcc None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:05.961634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c4622-1830-4c3a-8598-fd8de2cb3dcc None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:05.961634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 376/731] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:05.962733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af89ec1-e58b-4e18-ae48-883a3634802d None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:05.963021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af89ec1-e58b-4e18-ae48-883a3634802d None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:05.963329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af89ec1-e58b-4e18-ae48-883a3634802d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:05.966309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e15ba4-2d36-4bfb-b8a0-1b9ae11682eb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:05.981249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-9af89ec1-e58b-4e18-ae48-883a3634802d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:05.982466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9af89ec1-e58b-4e18-ae48-883a3634802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:06.004514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085f9fa-822c-4816-8b1b-35f1c17b250e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:06.005165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085f9fa-822c-4816-8b1b-35f1c17b250e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.005448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085f9fa-822c-4816-8b1b-35f1c17b250e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.005722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085f9fa-822c-4816-8b1b-35f1c17b250e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.064706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 377/732] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:01:06.064706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae72a3-a875-4241-b125-8557abb227e4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.064706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae72a3-a875-4241-b125-8557abb227e4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.064706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae72a3-a875-4241-b125-8557abb227e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.080321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-27ae72a3-a875-4241-b125-8557abb227e4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:06.082260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-27ae72a3-a875-4241-b125-8557abb227e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:06.087222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085f9fa-822c-4816-8b1b-35f1c17b250e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:06.088687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085f9fa-822c-4816-8b1b-35f1c17b250e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:06.152551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 378/733] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:01:06.153388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae72a3-a875-4241-b125-8557abb227e4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:06.170753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:06.182089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:06.194085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 356/734] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 05 15:01:06.195062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085f9fa-822c-4816-8b1b-35f1c17b250e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:06.255824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae72a3-a875-4241-b125-8557abb227e4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:06.288400 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33c36e-4a5a-452b-aac0-3bc704b56634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:06.289187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33c36e-4a5a-452b-aac0-3bc704b56634 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.289468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33c36e-4a5a-452b-aac0-3bc704b56634 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.289743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33c36e-4a5a-452b-aac0-3bc704b56634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.291082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085f9fa-822c-4816-8b1b-35f1c17b250e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:06.302062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33c36e-4a5a-452b-aac0-3bc704b56634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-af33c36e-4a5a-452b-aac0-3bc704b56634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:06.310276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:06.311230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 379/735] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:06.312811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc56640-6c3c-4c9a-84c9-b3309dfd1246 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.313561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc56640-6c3c-4c9a-84c9-b3309dfd1246 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.313561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc56640-6c3c-4c9a-84c9-b3309dfd1246 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.318870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 380/736] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:06.318870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc56640-6c3c-4c9a-84c9-b3309dfd1246 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:06.330077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b341b9e-62f8-4235-93ab-771b533e128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:06.330867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b341b9e-62f8-4235-93ab-771b533e128c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.331191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b341b9e-62f8-4235-93ab-771b533e128c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.331471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b341b9e-62f8-4235-93ab-771b533e128c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.342194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b341b9e-62f8-4235-93ab-771b533e128c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0b341b9e-62f8-4235-93ab-771b533e128c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:06.349712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:06.350811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 357/737] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:06.351889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b341b9e-62f8-4235-93ab-771b533e128c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:06.394130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:06.405437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:06.421329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc56640-6c3c-4c9a-84c9-b3309dfd1246 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:06.451580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b341b9e-62f8-4235-93ab-771b533e128c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:06.454165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8017d43-2fef-4ea7-8f23-78255ac21daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:06.454818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8017d43-2fef-4ea7-8f23-78255ac21daf None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.455128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8017d43-2fef-4ea7-8f23-78255ac21daf None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.455411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8017d43-2fef-4ea7-8f23-78255ac21daf None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.456175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8017d43-2fef-4ea7-8f23-78255ac21daf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:06.457565 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8017d43-2fef-4ea7-8f23-78255ac21daf None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:06.471175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 381/738] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:01:06 2022] POST /identity/v3/services => generated 251 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:06.471967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8017d43-2fef-4ea7-8f23-78255ac21daf None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:06.499527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51be231-7280-4f5c-9ac3-0b7718e6aa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:06.500407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51be231-7280-4f5c-9ac3-0b7718e6aa57 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.500688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51be231-7280-4f5c-9ac3-0b7718e6aa57 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.500994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51be231-7280-4f5c-9ac3-0b7718e6aa57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.512855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51be231-7280-4f5c-9ac3-0b7718e6aa57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:06.514362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51be231-7280-4f5c-9ac3-0b7718e6aa57 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b51be231-7280-4f5c-9ac3-0b7718e6aa57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:06.533159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:06.534165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:06.540327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:06.540327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:06.540327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:06.540327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:06.540327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 358/739] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:06.540327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0a7e6-3255-4393-a322-058b7d05275a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.540327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0a7e6-3255-4393-a322-058b7d05275a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.540327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0a7e6-3255-4393-a322-058b7d05275a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.555289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-65d0a7e6-3255-4393-a322-058b7d05275a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:06.556544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-65d0a7e6-3255-4393-a322-058b7d05275a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:06.601895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8017d43-2fef-4ea7-8f23-78255ac21daf None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:06.635998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 359/740] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 05 15:01:06.637326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0a7e6-3255-4393-a322-058b7d05275a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:06.639024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b977f3-2a02-401b-bb7d-6c6f2b68f427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:06.639746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b977f3-2a02-401b-bb7d-6c6f2b68f427 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.640035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b977f3-2a02-401b-bb7d-6c6f2b68f427 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.640310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b977f3-2a02-401b-bb7d-6c6f2b68f427 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.641062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b977f3-2a02-401b-bb7d-6c6f2b68f427 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:06.641370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b977f3-2a02-401b-bb7d-6c6f2b68f427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:06.642763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b977f3-2a02-401b-bb7d-6c6f2b68f427 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:06.653502 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 382/741] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:01:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:06.654704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b977f3-2a02-401b-bb7d-6c6f2b68f427 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:06.662734 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:06.670279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:06.745441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0a7e6-3255-4393-a322-058b7d05275a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:06.773049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b977f3-2a02-401b-bb7d-6c6f2b68f427 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:06.779073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60515-e274-4f29-b698-d6a7b48e732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:06.779747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60515-e274-4f29-b698-d6a7b48e732a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.780032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60515-e274-4f29-b698-d6a7b48e732a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.780309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c60515-e274-4f29-b698-d6a7b48e732a None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.781059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c60515-e274-4f29-b698-d6a7b48e732a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:06.781366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c60515-e274-4f29-b698-d6a7b48e732a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:06.782753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c60515-e274-4f29-b698-d6a7b48e732a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:06.795445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 360/742] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:01:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:06.795445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60515-e274-4f29-b698-d6a7b48e732a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:06.813940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9f414-2774-4555-a3ff-cf3517eecf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:06.814731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f414-2774-4555-a3ff-cf3517eecf43 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.815020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f414-2774-4555-a3ff-cf3517eecf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.815322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f414-2774-4555-a3ff-cf3517eecf43 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.816058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9f414-2774-4555-a3ff-cf3517eecf43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:06.816369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9f414-2774-4555-a3ff-cf3517eecf43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:06.818012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9f414-2774-4555-a3ff-cf3517eecf43 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:06.835864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 383/743] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:06.836544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9f414-2774-4555-a3ff-cf3517eecf43 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:06.876297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:06.899003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c60515-e274-4f29-b698-d6a7b48e732a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:06.903545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:06.935631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7271a9f-1fd5-498e-af2c-6f61813c6cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:06.936578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7271a9f-1fd5-498e-af2c-6f61813c6cfb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.937168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7271a9f-1fd5-498e-af2c-6f61813c6cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.937520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7271a9f-1fd5-498e-af2c-6f61813c6cfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.939582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9f414-2774-4555-a3ff-cf3517eecf43 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:06.948204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7271a9f-1fd5-498e-af2c-6f61813c6cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:06.950294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7271a9f-1fd5-498e-af2c-6f61813c6cfb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f7271a9f-1fd5-498e-af2c-6f61813c6cfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:06.968567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:06.969473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:06.970371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:06.970371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:06.970371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:06.970371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:06.970371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 361/744] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:06.971257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7271a9f-1fd5-498e-af2c-6f61813c6cfb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:06.978818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b573f-2b55-458b-a535-73ade1b9e85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:06.979518 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b573f-2b55-458b-a535-73ade1b9e85b None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:06.980303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b573f-2b55-458b-a535-73ade1b9e85b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:06.980585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b573f-2b55-458b-a535-73ade1b9e85b None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:06.981310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b573f-2b55-458b-a535-73ade1b9e85b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:06.982746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b573f-2b55-458b-a535-73ade1b9e85b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:06.996793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 384/745] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:01:06 2022] POST /identity/v3/services => generated 240 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:06.997594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b573f-2b55-458b-a535-73ade1b9e85b None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:07.095161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7271a9f-1fd5-498e-af2c-6f61813c6cfb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:07.126846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b573f-2b55-458b-a535-73ade1b9e85b None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:07.138492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c95ecf-c538-4c13-8f60-7cdb297cf0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:07.139526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c95ecf-c538-4c13-8f60-7cdb297cf0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:07.139649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c95ecf-c538-4c13-8f60-7cdb297cf0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:07.139905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c95ecf-c538-4c13-8f60-7cdb297cf0c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:07.142025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:07.153507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c95ecf-c538-4c13-8f60-7cdb297cf0c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:07.155327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c95ecf-c538-4c13-8f60-7cdb297cf0c8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:07.155451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:07.170134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f55f9-45c0-4004-9dde-62871407e2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:07.170942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f55f9-45c0-4004-9dde-62871407e2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:07.171261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f55f9-45c0-4004-9dde-62871407e2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:07.171557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f55f9-45c0-4004-9dde-62871407e2b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e4c95ecf-c538-4c13-8f60-7cdb297cf0c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:07.178822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:07.180077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:07.181166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:07.181166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:07.181166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:07.181166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:07.181448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f55f9-45c0-4004-9dde-62871407e2b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:07.181448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106f55f9-45c0-4004-9dde-62871407e2b5 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:07.187921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 362/746] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:07.189115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c95ecf-c538-4c13-8f60-7cdb297cf0c8 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-106f55f9-45c0-4004-9dde-62871407e2b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:07.196618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:07.197617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:07.198613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:07.198613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:07.198613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:07.198613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:07.198613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:07.198613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:07.198613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:07.198613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:07.203724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 385/747] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:07.204701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f55f9-45c0-4004-9dde-62871407e2b5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:07.317359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c95ecf-c538-4c13-8f60-7cdb297cf0c8 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:07.324664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f55f9-45c0-4004-9dde-62871407e2b5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:07.353279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e317b-330d-4988-960f-07dc7dfd5141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:07.354725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e317b-330d-4988-960f-07dc7dfd5141 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:07.355330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e317b-330d-4988-960f-07dc7dfd5141 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:07.355832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0e317b-330d-4988-960f-07dc7dfd5141 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:07.356764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e317b-330d-4988-960f-07dc7dfd5141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:07.357276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e317b-330d-4988-960f-07dc7dfd5141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:07.359157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0e317b-330d-4988-960f-07dc7dfd5141 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:07.361069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedfd34-9a2e-48a0-8998-da964b6432bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:07.362198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedfd34-9a2e-48a0-8998-da964b6432bb None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:07.362198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedfd34-9a2e-48a0-8998-da964b6432bb None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:07.362437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbedfd34-9a2e-48a0-8998-da964b6432bb None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:07.363351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedfd34-9a2e-48a0-8998-da964b6432bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:07.363745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedfd34-9a2e-48a0-8998-da964b6432bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:07.365226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbedfd34-9a2e-48a0-8998-da964b6432bb None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:07.373492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 363/748] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:07.374573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e317b-330d-4988-960f-07dc7dfd5141 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:07.377850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 386/749] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:07.382839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedfd34-9a2e-48a0-8998-da964b6432bb None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:07.423417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:07.455469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:07.478738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0e317b-330d-4988-960f-07dc7dfd5141 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:07.491703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbedfd34-9a2e-48a0-8998-da964b6432bb None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:07.514783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063ba03-d76c-485e-9d59-f3edd455d197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:07.515586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063ba03-d76c-485e-9d59-f3edd455d197 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:07.515947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063ba03-d76c-485e-9d59-f3edd455d197 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:07.516298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063ba03-d76c-485e-9d59-f3edd455d197 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:07.517203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063ba03-d76c-485e-9d59-f3edd455d197 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:07.517622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063ba03-d76c-485e-9d59-f3edd455d197 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:07.519370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063ba03-d76c-485e-9d59-f3edd455d197 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:07.527717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35e17f-dc69-4d6f-b295-351a868bb149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:07.528517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35e17f-dc69-4d6f-b295-351a868bb149 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:07.528795 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35e17f-dc69-4d6f-b295-351a868bb149 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:07.529070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35e17f-dc69-4d6f-b295-351a868bb149 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:07.533273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 364/750] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:07.534260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063ba03-d76c-485e-9d59-f3edd455d197 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:07.542636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35e17f-dc69-4d6f-b295-351a868bb149 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:07.544161 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35e17f-dc69-4d6f-b295-351a868bb149 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-ab35e17f-dc69-4d6f-b295-351a868bb149 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:07.560084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:07.561054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 387/751] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 5 15:01:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:07.562429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35e17f-dc69-4d6f-b295-351a868bb149 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:07.658888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063ba03-d76c-485e-9d59-f3edd455d197 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:07.666580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:07.693817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35e17f-dc69-4d6f-b295-351a868bb149 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:07.695537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59980399-f04f-4928-8eb8-926b9f6672fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:07.696405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59980399-f04f-4928-8eb8-926b9f6672fc None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:07.696561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:07.696810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59980399-f04f-4928-8eb8-926b9f6672fc None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:07.697151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59980399-f04f-4928-8eb8-926b9f6672fc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:07.706147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59980399-f04f-4928-8eb8-926b9f6672fc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:07.707720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59980399-f04f-4928-8eb8-926b9f6672fc None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-59980399-f04f-4928-8eb8-926b9f6672fc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:07.722142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:07.724033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:07.726294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:07.726294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:07.726294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:07.726294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:07.726294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:07.726294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:07.726294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:07.726294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:07.726294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:01:07.726294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:07.728896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 365/752] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:07.731091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59980399-f04f-4928-8eb8-926b9f6672fc None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:07.735616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fcb17-f24c-4a91-b9c9-a6b987c618fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:07.736528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fcb17-f24c-4a91-b9c9-a6b987c618fc None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:07.736963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fcb17-f24c-4a91-b9c9-a6b987c618fc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:07.737439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fcb17-f24c-4a91-b9c9-a6b987c618fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:07.738840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fcb17-f24c-4a91-b9c9-a6b987c618fc None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=b3ab5ed3d9e341f7853302ac7826879b) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:07.739340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fcb17-f24c-4a91-b9c9-a6b987c618fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:07.741001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fcb17-f24c-4a91-b9c9-a6b987c618fc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:07.749451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 388/753] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:07 2022] GET /identity/v3/role_assignments?user.id=b3ab5ed3d9e341f7853302ac7826879b&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:07.750507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fcb17-f24c-4a91-b9c9-a6b987c618fc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:08.007576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59980399-f04f-4928-8eb8-926b9f6672fc None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:08.009220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fcb17-f24c-4a91-b9c9-a6b987c618fc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:08.059885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eedd7-bc5c-4df2-a40d-696457656e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:08.060571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eedd7-bc5c-4df2-a40d-696457656e6a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:08.060858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eedd7-bc5c-4df2-a40d-696457656e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:08.061138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8eedd7-bc5c-4df2-a40d-696457656e6a None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:08.061995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eedd7-bc5c-4df2-a40d-696457656e6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:08.062309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eedd7-bc5c-4df2-a40d-696457656e6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:08.063802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8eedd7-bc5c-4df2-a40d-696457656e6a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:08.066617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57609b0e-7d82-4acc-801e-73d80f5038e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:08.067317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57609b0e-7d82-4acc-801e-73d80f5038e0 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:08.067605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57609b0e-7d82-4acc-801e-73d80f5038e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:08.067884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57609b0e-7d82-4acc-801e-73d80f5038e0 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:08.068733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57609b0e-7d82-4acc-801e-73d80f5038e0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:08.069047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57609b0e-7d82-4acc-801e-73d80f5038e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:08.070495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57609b0e-7d82-4acc-801e-73d80f5038e0 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:08.101461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 389/754] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 5 15:01:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:08.103954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 366/755] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 5 15:01:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:08.108186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57609b0e-7d82-4acc-801e-73d80f5038e0 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:08.112797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eedd7-bc5c-4df2-a40d-696457656e6a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:08.142871 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:08.165126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:08.281054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57609b0e-7d82-4acc-801e-73d80f5038e0 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:08.303880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8eedd7-bc5c-4df2-a40d-696457656e6a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:08.348699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f4e60-1c05-43c2-bc72-8d5a5f71c37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:08.349372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f4e60-1c05-43c2-bc72-8d5a5f71c37d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:08.349665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f4e60-1c05-43c2-bc72-8d5a5f71c37d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:08.349949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f4e60-1c05-43c2-bc72-8d5a5f71c37d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:08.355802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae02161-2f1f-4681-99eb-e09238838f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:08.356705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae02161-2f1f-4681-99eb-e09238838f12 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:08.357080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae02161-2f1f-4681-99eb-e09238838f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:08.357473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae02161-2f1f-4681-99eb-e09238838f12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:08.368168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae02161-2f1f-4681-99eb-e09238838f12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:08.369728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae02161-2f1f-4681-99eb-e09238838f12 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-5ae02161-2f1f-4681-99eb-e09238838f12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:08.396769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:08.397784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:08.398683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:08.398683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:08.398683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:08.398683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:08.398683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 367/756] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:08.404691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae02161-2f1f-4681-99eb-e09238838f12 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:08.473167 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f4e60-1c05-43c2-bc72-8d5a5f71c37d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:08.474639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f4e60-1c05-43c2-bc72-8d5a5f71c37d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:08.526119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:08.550693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:08.585530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae02161-2f1f-4681-99eb-e09238838f12 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:08.622200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 390/757] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:01:08.627596 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f4e60-1c05-43c2-bc72-8d5a5f71c37d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:08.642273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e1aad-bc3b-41c9-bf0f-0e4ece0172d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:08.642940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e1aad-bc3b-41c9-bf0f-0e4ece0172d5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:08.643253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e1aad-bc3b-41c9-bf0f-0e4ece0172d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:08.643532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e1aad-bc3b-41c9-bf0f-0e4ece0172d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:08.644267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e1aad-bc3b-41c9-bf0f-0e4ece0172d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:08.644576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e1aad-bc3b-41c9-bf0f-0e4ece0172d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:08.646073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e1aad-bc3b-41c9-bf0f-0e4ece0172d5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:08.659145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 368/758] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:08.664146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e1aad-bc3b-41c9-bf0f-0e4ece0172d5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:08.767679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f4e60-1c05-43c2-bc72-8d5a5f71c37d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:08.795702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:08.806220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20635b-b8c7-41c2-b14e-1caed8108aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:08.807321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20635b-b8c7-41c2-b14e-1caed8108aed None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:08.807845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20635b-b8c7-41c2-b14e-1caed8108aed None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:08.808347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20635b-b8c7-41c2-b14e-1caed8108aed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:08.813287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e1aad-bc3b-41c9-bf0f-0e4ece0172d5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:08.819968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20635b-b8c7-41c2-b14e-1caed8108aed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:08.821799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20635b-b8c7-41c2-b14e-1caed8108aed None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-bb20635b-b8c7-41c2-b14e-1caed8108aed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:08.841976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:08.842909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:08.843863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:08.843863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:08.843863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:08.843863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:08.850377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 391/759] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:08.851949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0a0e2-18a0-4dad-8bf9-dd753fefb81e None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:08.852598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0a0e2-18a0-4dad-8bf9-dd753fefb81e None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:08.853064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b0a0e2-18a0-4dad-8bf9-dd753fefb81e None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:08.853822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b006888-c021-42cf-a23d-e4615a04a484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:08.854813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006888-c021-42cf-a23d-e4615a04a484 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:08.855156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 392/760] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:08.855270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006888-c021-42cf-a23d-e4615a04a484 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:08.855658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b006888-c021-42cf-a23d-e4615a04a484 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:08.856287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0a0e2-18a0-4dad-8bf9-dd753fefb81e None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:08.862687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:08.868094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b006888-c021-42cf-a23d-e4615a04a484 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:08.869813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b006888-c021-42cf-a23d-e4615a04a484 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4b006888-c021-42cf-a23d-e4615a04a484 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:08.885657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:08.886644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:08.887906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:08.887906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:08.887906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:08.887906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:08.887906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:08.887906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:08.887906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:08.887906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:08.890234 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 369/761] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:08.891986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee625a-e031-4800-a211-4edc50c27cba None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:08.892495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee625a-e031-4800-a211-4edc50c27cba None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:08.892970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee625a-e031-4800-a211-4edc50c27cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:08.911960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fbee625a-e031-4800-a211-4edc50c27cba None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:08.914106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fbee625a-e031-4800-a211-4edc50c27cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:08.982653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b0a0e2-18a0-4dad-8bf9-dd753fefb81e None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:08.995373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 370/762] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:08.996705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee625a-e031-4800-a211-4edc50c27cba None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:09.023143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52126ea8-5d14-476e-b87f-50134642fbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:09.024562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52126ea8-5d14-476e-b87f-50134642fbba None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.024873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52126ea8-5d14-476e-b87f-50134642fbba None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.025159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52126ea8-5d14-476e-b87f-50134642fbba None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.026009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52126ea8-5d14-476e-b87f-50134642fbba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:09.026370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52126ea8-5d14-476e-b87f-50134642fbba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:09.028274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52126ea8-5d14-476e-b87f-50134642fbba None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:09.047084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 393/763] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:09.047084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52126ea8-5d14-476e-b87f-50134642fbba None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:09.100693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:09.129064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee625a-e031-4800-a211-4edc50c27cba None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:09.158278 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:09.169445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4277b6-ce89-40b9-bc4f-86b494fcd701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:09.170529 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4277b6-ce89-40b9-bc4f-86b494fcd701 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.171105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4277b6-ce89-40b9-bc4f-86b494fcd701 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.171590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4277b6-ce89-40b9-bc4f-86b494fcd701 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.172644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4277b6-ce89-40b9-bc4f-86b494fcd701 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:09.173269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4277b6-ce89-40b9-bc4f-86b494fcd701 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:09.175006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4277b6-ce89-40b9-bc4f-86b494fcd701 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:09.180114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52126ea8-5d14-476e-b87f-50134642fbba None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:09.194653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 371/764] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:09.195541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4277b6-ce89-40b9-bc4f-86b494fcd701 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:09.223189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7754142-a3ec-4e9a-a2ae-e479996ec2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:09.224468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7754142-a3ec-4e9a-a2ae-e479996ec2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.224962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7754142-a3ec-4e9a-a2ae-e479996ec2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.225494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7754142-a3ec-4e9a-a2ae-e479996ec2e6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.226397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7754142-a3ec-4e9a-a2ae-e479996ec2e6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:09.228137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7754142-a3ec-4e9a-a2ae-e479996ec2e6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-c7754142-a3ec-4e9a-a2ae-e479996ec2e6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:09.241314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:09.242352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:01:09.243234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:09.247113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 394/765] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Jul 5 15:01:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:09.248382 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7754142-a3ec-4e9a-a2ae-e479996ec2e6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:09.314400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4277b6-ce89-40b9-bc4f-86b494fcd701 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:09.349180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7754142-a3ec-4e9a-a2ae-e479996ec2e6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:09.351114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:09.354990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7047c31-af6d-4375-847b-836164bf4545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:09.356014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7047c31-af6d-4375-847b-836164bf4545 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.356525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7047c31-af6d-4375-847b-836164bf4545 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.356602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7047c31-af6d-4375-847b-836164bf4545 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.364816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7047c31-af6d-4375-847b-836164bf4545 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:09.366293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7047c31-af6d-4375-847b-836164bf4545 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c7047c31-af6d-4375-847b-836164bf4545 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:09.381037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:09.381947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:09.382876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:09.382876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:09.382876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:09.382876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:09.382876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:09.382876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:09.382876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:09.382876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:09.382876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 372/766] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:09.383961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf533f-bd39-4414-9e7b-6439830e5b85 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.383961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf533f-bd39-4414-9e7b-6439830e5b85 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.383961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf533f-bd39-4414-9e7b-6439830e5b85 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.385368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 373/767] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:09.385963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf533f-bd39-4414-9e7b-6439830e5b85 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:09.394682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff5607-70d0-40e7-b61b-4f313eebea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:09.395401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff5607-70d0-40e7-b61b-4f313eebea6c None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.395687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff5607-70d0-40e7-b61b-4f313eebea6c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.395963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff5607-70d0-40e7-b61b-4f313eebea6c None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/e7a416f22d33450183801cef2b13822f/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.398095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff5607-70d0-40e7-b61b-4f313eebea6c None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=e7a416f22d33450183801cef2b13822f, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:09.399637 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bff5607-70d0-40e7-b61b-4f313eebea6c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:09.427074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 395/768] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:01:09 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/e7a416f22d33450183801cef2b13822f/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:01:09.427074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff5607-70d0-40e7-b61b-4f313eebea6c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:09.448673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:09.519526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adf533f-bd39-4414-9e7b-6439830e5b85 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:09.570269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff5607-70d0-40e7-b61b-4f313eebea6c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:09.570473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab7fa0-bd75-42ff-ab50-13a0da3bdb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:09.570989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7fa0-bd75-42ff-ab50-13a0da3bdb34 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.571309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7fa0-bd75-42ff-ab50-13a0da3bdb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.571594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab7fa0-bd75-42ff-ab50-13a0da3bdb34 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.572323 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab7fa0-bd75-42ff-ab50-13a0da3bdb34 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:09.572672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab7fa0-bd75-42ff-ab50-13a0da3bdb34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:09.576243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab7fa0-bd75-42ff-ab50-13a0da3bdb34 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:09.582113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 374/769] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 5 15:01:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:09.583705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95081fa9-1a0d-4f1c-920c-1a444761ce68 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.584008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95081fa9-1a0d-4f1c-920c-1a444761ce68 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.584292 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95081fa9-1a0d-4f1c-920c-1a444761ce68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.606195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-95081fa9-1a0d-4f1c-920c-1a444761ce68 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:09.607369 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-95081fa9-1a0d-4f1c-920c-1a444761ce68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:09.610642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039be9a4-94b4-4869-bf9d-13037cd9b39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:09.611372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039be9a4-94b4-4869-bf9d-13037cd9b39a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.611661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039be9a4-94b4-4869-bf9d-13037cd9b39a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.611942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039be9a4-94b4-4869-bf9d-13037cd9b39a None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.613701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039be9a4-94b4-4869-bf9d-13037cd9b39a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:09.614022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039be9a4-94b4-4869-bf9d-13037cd9b39a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:09.615509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039be9a4-94b4-4869-bf9d-13037cd9b39a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:09.629830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 396/770] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:09.630696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039be9a4-94b4-4869-bf9d-13037cd9b39a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:09.657398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:09.713343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 375/771] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:09.714620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95081fa9-1a0d-4f1c-920c-1a444761ce68 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:09.741776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039be9a4-94b4-4869-bf9d-13037cd9b39a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:09.769400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:09.779973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74325f-abc6-4060-806d-f6f5f1ce82c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:09.780653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74325f-abc6-4060-806d-f6f5f1ce82c2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.780944 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74325f-abc6-4060-806d-f6f5f1ce82c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.781226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74325f-abc6-4060-806d-f6f5f1ce82c2 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.781955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74325f-abc6-4060-806d-f6f5f1ce82c2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:09.782455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74325f-abc6-4060-806d-f6f5f1ce82c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:09.783687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74325f-abc6-4060-806d-f6f5f1ce82c2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:09.789101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 397/772] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 5 15:01:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:09.789917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74325f-abc6-4060-806d-f6f5f1ce82c2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:09.840434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95081fa9-1a0d-4f1c-920c-1a444761ce68 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:09.882600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac7ab5-060b-4c20-be40-d4a6838a2dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:09.883653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac7ab5-060b-4c20-be40-d4a6838a2dcb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.884561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac7ab5-060b-4c20-be40-d4a6838a2dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.885040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac7ab5-060b-4c20-be40-d4a6838a2dcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.886958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac7ab5-060b-4c20-be40-d4a6838a2dcb None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=4c73aa1f244d4251b8539c2806f38a42) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:09.887509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac7ab5-060b-4c20-be40-d4a6838a2dcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:09.889775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac7ab5-060b-4c20-be40-d4a6838a2dcb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:09.895076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:09.897401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 376/773] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:09 2022] GET /identity/v3/role_assignments?user.id=4c73aa1f244d4251b8539c2806f38a42&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:09.898310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac7ab5-060b-4c20-be40-d4a6838a2dcb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:09.915880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74325f-abc6-4060-806d-f6f5f1ce82c2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:09.958829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b146257a-bfe6-481e-93aa-9689472fbaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:09.960079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146257a-bfe6-481e-93aa-9689472fbaea None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.960881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146257a-bfe6-481e-93aa-9689472fbaea None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.961172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b146257a-bfe6-481e-93aa-9689472fbaea None admin] PATH_INFO: `/v3/services/network` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.961849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146257a-bfe6-481e-93aa-9689472fbaea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:09.963306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b146257a-bfe6-481e-93aa-9689472fbaea None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-b146257a-bfe6-481e-93aa-9689472fbaea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:09.976389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:09.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:09.978894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 398/774] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:09 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:09.980287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeedd1-f597-4a89-bad1-7479d3a05458 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:09.980364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeedd1-f597-4a89-bad1-7479d3a05458 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:09.980659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aeedd1-f597-4a89-bad1-7479d3a05458 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:09.982886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 399/775] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:09.985982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeedd1-f597-4a89-bad1-7479d3a05458 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:10.037926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac7ab5-060b-4c20-be40-d4a6838a2dcb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:10.041368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:10.086834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b875d-e000-49eb-8355-24d55572e3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:10.087791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b875d-e000-49eb-8355-24d55572e3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:10.088689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b875d-e000-49eb-8355-24d55572e3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:10.089697 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b875d-e000-49eb-8355-24d55572e3e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:10.090963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b875d-e000-49eb-8355-24d55572e3e2 None admin] RBAC: Adding query filter params (interface=public, service_id=2b2e66996e76460b8cf5ae62b6a675a7, region_id=RegionOne) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:10.091518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b875d-e000-49eb-8355-24d55572e3e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:10.093579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b875d-e000-49eb-8355-24d55572e3e2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:10.103803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 377/776] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 5 15:01:09 2022] GET /identity/v3/endpoints?service_id=2b2e66996e76460b8cf5ae62b6a675a7&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:10.105279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12121e3-ad1a-4065-b70f-6d5678bf810e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:10.105718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12121e3-ad1a-4065-b70f-6d5678bf810e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:10.106164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12121e3-ad1a-4065-b70f-6d5678bf810e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:10.109840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aeedd1-f597-4a89-bad1-7479d3a05458 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:10.127312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f12121e3-ad1a-4065-b70f-6d5678bf810e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:10.128908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f12121e3-ad1a-4065-b70f-6d5678bf810e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:10.146474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e594d4-bfb7-4c06-9e75-e5e21c43a152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:10.147094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e594d4-bfb7-4c06-9e75-e5e21c43a152 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:10.147388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e594d4-bfb7-4c06-9e75-e5e21c43a152 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:10.147669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e594d4-bfb7-4c06-9e75-e5e21c43a152 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:10.148392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e594d4-bfb7-4c06-9e75-e5e21c43a152 None admin] RBAC: Adding query filter params (name=network) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:10.148702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e594d4-bfb7-4c06-9e75-e5e21c43a152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:10.150278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e594d4-bfb7-4c06-9e75-e5e21c43a152 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:10.155428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 400/777] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:10.156311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e594d4-bfb7-4c06-9e75-e5e21c43a152 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:10.179473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:10.206026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 378/778] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:10.207272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12121e3-ad1a-4065-b70f-6d5678bf810e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:10.272009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e594d4-bfb7-4c06-9e75-e5e21c43a152 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:10.312973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12121e3-ad1a-4065-b70f-6d5678bf810e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:10.313333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496277a-4bd2-4270-a5dc-4c0f6d60ea75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:10.314001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496277a-4bd2-4270-a5dc-4c0f6d60ea75 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:10.314290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496277a-4bd2-4270-a5dc-4c0f6d60ea75 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:10.314568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496277a-4bd2-4270-a5dc-4c0f6d60ea75 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:10.315331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496277a-4bd2-4270-a5dc-4c0f6d60ea75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:10.316105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:10.318478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496277a-4bd2-4270-a5dc-4c0f6d60ea75 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:10.331082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 401/779] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 5 15:01:10 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:10.331082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496277a-4bd2-4270-a5dc-4c0f6d60ea75 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:10.383027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a66c0e-d269-4658-b4c4-b939a3235820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:10.383737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a66c0e-d269-4658-b4c4-b939a3235820 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:10.384000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a66c0e-d269-4658-b4c4-b939a3235820 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:10.384263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a66c0e-d269-4658-b4c4-b939a3235820 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:10.384988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a66c0e-d269-4658-b4c4-b939a3235820 None admin] RBAC: Adding query filter params (type=network) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:10.385294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a66c0e-d269-4658-b4c4-b939a3235820 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:10.387300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a66c0e-d269-4658-b4c4-b939a3235820 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:10.400639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 379/780] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:10.410397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a66c0e-d269-4658-b4c4-b939a3235820 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:10.483553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:10.509525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496277a-4bd2-4270-a5dc-4c0f6d60ea75 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:10.546669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad7d7a-860e-4d6d-b574-b1dd51d86a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:10.547580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad7d7a-860e-4d6d-b574-b1dd51d86a74 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:10.547870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad7d7a-860e-4d6d-b574-b1dd51d86a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:10.548151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad7d7a-860e-4d6d-b574-b1dd51d86a74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:10.562879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad7d7a-860e-4d6d-b574-b1dd51d86a74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-c8ad7d7a-860e-4d6d-b574-b1dd51d86a74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:10.573933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:10.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:01:10.576055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:10.577306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 402/781] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:01:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:10.582396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad7d7a-860e-4d6d-b574-b1dd51d86a74 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:10.584736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a66c0e-d269-4658-b4c4-b939a3235820 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:10.630774 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf89c5-f817-4f7e-ac41-6545e738daec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:10.631837 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf89c5-f817-4f7e-ac41-6545e738daec None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:10.632128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf89c5-f817-4f7e-ac41-6545e738daec None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:10.632414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf89c5-f817-4f7e-ac41-6545e738daec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:10.633537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf89c5-f817-4f7e-ac41-6545e738daec None admin] RBAC: Adding query filter params (interface=public, service_id=dd35331a079e415b883053338a31f934, region_id=RegionOne) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:10.633866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf89c5-f817-4f7e-ac41-6545e738daec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:10.635591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf89c5-f817-4f7e-ac41-6545e738daec None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:10.643232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 380/782] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 5 15:01:10 2022] GET /identity/v3/endpoints?service_id=dd35331a079e415b883053338a31f934&interface=public®ion_id=RegionOne => generated 194 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:10.652248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf89c5-f817-4f7e-ac41-6545e738daec None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:10.694854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:10.713615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad7d7a-860e-4d6d-b574-b1dd51d86a74 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:10.758865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cf288-737a-407e-b4aa-dfe5c40128a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:10.759584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf288-737a-407e-b4aa-dfe5c40128a0 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:10.759917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf288-737a-407e-b4aa-dfe5c40128a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:10.760202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17cf288-737a-407e-b4aa-dfe5c40128a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:10.761514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cf288-737a-407e-b4aa-dfe5c40128a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:10.761826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cf288-737a-407e-b4aa-dfe5c40128a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:10.767290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17cf288-737a-407e-b4aa-dfe5c40128a0 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:10.775398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf89c5-f817-4f7e-ac41-6545e738daec None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:10.779400 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:10.783572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 403/783] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:01:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:10.788991 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cf288-737a-407e-b4aa-dfe5c40128a0 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:10.811494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e13b9-0b4b-4db2-b630-262415aa9d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:10.812168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e13b9-0b4b-4db2-b630-262415aa9d71 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:10.812453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e13b9-0b4b-4db2-b630-262415aa9d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:10.812732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e13b9-0b4b-4db2-b630-262415aa9d71 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:10.813420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e13b9-0b4b-4db2-b630-262415aa9d71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:10.814798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e13b9-0b4b-4db2-b630-262415aa9d71 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:10.820676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 381/784] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 5 15:01:10 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:10.963611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17cf288-737a-407e-b4aa-dfe5c40128a0 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:11.000178 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460564a2-ffab-4dcd-9077-03e0c003e654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:11.001052 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460564a2-ffab-4dcd-9077-03e0c003e654 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:11.001342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460564a2-ffab-4dcd-9077-03e0c003e654 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:11.001620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460564a2-ffab-4dcd-9077-03e0c003e654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:11.013694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460564a2-ffab-4dcd-9077-03e0c003e654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:11.015214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460564a2-ffab-4dcd-9077-03e0c003e654 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-460564a2-ffab-4dcd-9077-03e0c003e654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:11.035406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:11.036523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:11.037629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:11.037629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:11.037629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:11.037629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:11.037629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 404/785] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:11.042454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e13b9-0b4b-4db2-b630-262415aa9d71 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:11.161775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:11.172391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e13b9-0b4b-4db2-b630-262415aa9d71 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:11.223769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cac57-243f-4a84-9236-d65199beea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:11.224363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cac57-243f-4a84-9236-d65199beea9c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:11.224645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cac57-243f-4a84-9236-d65199beea9c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:11.224920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cac57-243f-4a84-9236-d65199beea9c None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:11.225653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cac57-243f-4a84-9236-d65199beea9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:11.225957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cac57-243f-4a84-9236-d65199beea9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:11.227340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cac57-243f-4a84-9236-d65199beea9c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:11.243952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 382/786] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:11.249552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460564a2-ffab-4dcd-9077-03e0c003e654 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:11.337246 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:11.386939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460564a2-ffab-4dcd-9077-03e0c003e654 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:11.427778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b44b05-6ed4-48e7-bbd4-ff4e2196f0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:11.428663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44b05-6ed4-48e7-bbd4-ff4e2196f0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:11.428959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44b05-6ed4-48e7-bbd4-ff4e2196f0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:11.429241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b44b05-6ed4-48e7-bbd4-ff4e2196f0a6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:11.437201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b44b05-6ed4-48e7-bbd4-ff4e2196f0a6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:11.438651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b44b05-6ed4-48e7-bbd4-ff4e2196f0a6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-60b44b05-6ed4-48e7-bbd4-ff4e2196f0a6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:11.453929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:11.454873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:11.455857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 405/787] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 5 15:01:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:11.460833 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cac57-243f-4a84-9236-d65199beea9c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:11.599935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cac57-243f-4a84-9236-d65199beea9c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:11.644351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd5f53-de1e-4b71-9534-0be4b5d7e068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:11.645034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5f53-de1e-4b71-9534-0be4b5d7e068 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:11.645314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5f53-de1e-4b71-9534-0be4b5d7e068 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:11.645583 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd5f53-de1e-4b71-9534-0be4b5d7e068 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:11.646433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd5f53-de1e-4b71-9534-0be4b5d7e068 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:11.646742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd5f53-de1e-4b71-9534-0be4b5d7e068 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:11.648215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd5f53-de1e-4b71-9534-0be4b5d7e068 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:11.674606 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 383/788] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 5 15:01:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:11.680045 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b44b05-6ed4-48e7-bbd4-ff4e2196f0a6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:11.786368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef1b3-a7cf-44ff-b350-72091213ee98 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:11.786692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef1b3-a7cf-44ff-b350-72091213ee98 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:11.786932 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ef1b3-a7cf-44ff-b350-72091213ee98 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:11.788611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 384/789] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:11.793824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec93c74-623b-4e95-9a2d-81e1906f0298 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:11.794104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec93c74-623b-4e95-9a2d-81e1906f0298 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:11.794383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec93c74-623b-4e95-9a2d-81e1906f0298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:11.823371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cec93c74-623b-4e95-9a2d-81e1906f0298 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:11.824528 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cec93c74-623b-4e95-9a2d-81e1906f0298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:11.826814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b44b05-6ed4-48e7-bbd4-ff4e2196f0a6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:11.827325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:11.840569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:11.887905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028e3f4-02ca-45e0-8b04-896c7f1b68b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:11.888949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028e3f4-02ca-45e0-8b04-896c7f1b68b4 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:11.889486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028e3f4-02ca-45e0-8b04-896c7f1b68b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:11.889919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028e3f4-02ca-45e0-8b04-896c7f1b68b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:11.911266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a028e3f4-02ca-45e0-8b04-896c7f1b68b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:11.913043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a028e3f4-02ca-45e0-8b04-896c7f1b68b4 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-a028e3f4-02ca-45e0-8b04-896c7f1b68b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:11.940294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:11.941421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:11.942514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:11.942514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:11.942514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:11.942514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:11.956213 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 385/791] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:11.956481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 406/791] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:11.957842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec93c74-623b-4e95-9a2d-81e1906f0298 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:12.000703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028e3f4-02ca-45e0-8b04-896c7f1b68b4 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:12.110752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec93c74-623b-4e95-9a2d-81e1906f0298 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:12.172037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028e3f4-02ca-45e0-8b04-896c7f1b68b4 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:12.175479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864879c-d75a-42aa-a909-e27b9001b432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:12.176393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864879c-d75a-42aa-a909-e27b9001b432 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.177615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864879c-d75a-42aa-a909-e27b9001b432 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.178051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1864879c-d75a-42aa-a909-e27b9001b432 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.182886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864879c-d75a-42aa-a909-e27b9001b432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:12.183370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864879c-d75a-42aa-a909-e27b9001b432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:12.184876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1864879c-d75a-42aa-a909-e27b9001b432 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:12.194703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:12.208117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 386/792] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:12.214140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864879c-d75a-42aa-a909-e27b9001b432 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:12.263771 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8caf515-7fe9-4ed9-979f-7342b04839d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:12.264592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caf515-7fe9-4ed9-979f-7342b04839d3 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.264888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caf515-7fe9-4ed9-979f-7342b04839d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.265176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8caf515-7fe9-4ed9-979f-7342b04839d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.285733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8caf515-7fe9-4ed9-979f-7342b04839d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-d8caf515-7fe9-4ed9-979f-7342b04839d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:12.298321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:12.299339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:12.300194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:12.316243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 407/793] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:12.320634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8caf515-7fe9-4ed9-979f-7342b04839d3 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:12.367848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:12.380414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1864879c-d75a-42aa-a909-e27b9001b432 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:12.444948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334ef03-f0f1-4d23-8796-d31a8cb1fefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:12.446118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334ef03-f0f1-4d23-8796-d31a8cb1fefa None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.446599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334ef03-f0f1-4d23-8796-d31a8cb1fefa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.447091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334ef03-f0f1-4d23-8796-d31a8cb1fefa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.462259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334ef03-f0f1-4d23-8796-d31a8cb1fefa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:12.465044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334ef03-f0f1-4d23-8796-d31a8cb1fefa None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-8334ef03-f0f1-4d23-8796-d31a8cb1fefa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:12.486391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:12.487349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:12.488284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:12.488284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:12.488284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:12.488284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:12.488284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:12.488284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:12.488284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:12.488284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:12.495649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 387/794] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:12.497055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334ef03-f0f1-4d23-8796-d31a8cb1fefa None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:12.514336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8caf515-7fe9-4ed9-979f-7342b04839d3 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:12.551914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4372b02-ce18-45de-9a20-5763c1e3c94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:12.552913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4372b02-ce18-45de-9a20-5763c1e3c94f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.553193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4372b02-ce18-45de-9a20-5763c1e3c94f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.553471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4372b02-ce18-45de-9a20-5763c1e3c94f None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.554240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4372b02-ce18-45de-9a20-5763c1e3c94f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:12.554553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4372b02-ce18-45de-9a20-5763c1e3c94f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:12.555983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4372b02-ce18-45de-9a20-5763c1e3c94f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:12.569157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 408/795] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:12.569157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a79721-4112-452d-afee-dcdb8e61f963 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.569157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a79721-4112-452d-afee-dcdb8e61f963 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.569157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a79721-4112-452d-afee-dcdb8e61f963 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.569157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 409/796] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:12.574779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee4a1d-f5e8-4270-8284-8be177768052 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.575000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee4a1d-f5e8-4270-8284-8be177768052 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.575286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee4a1d-f5e8-4270-8284-8be177768052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.583028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:12.595945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-f6ee4a1d-f5e8-4270-8284-8be177768052 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:12.597311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f6ee4a1d-f5e8-4270-8284-8be177768052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:12.637596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334ef03-f0f1-4d23-8796-d31a8cb1fefa None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:12.681395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63dff8-334a-44ad-b8a1-34bf5447b11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:12.682970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63dff8-334a-44ad-b8a1-34bf5447b11f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.683491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63dff8-334a-44ad-b8a1-34bf5447b11f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.684058 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63dff8-334a-44ad-b8a1-34bf5447b11f None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.685057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63dff8-334a-44ad-b8a1-34bf5447b11f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:12.685569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63dff8-334a-44ad-b8a1-34bf5447b11f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:12.687225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63dff8-334a-44ad-b8a1-34bf5447b11f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:12.688469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 410/797] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:12.689345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee4a1d-f5e8-4270-8284-8be177768052 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:12.707353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 388/798] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:12.708901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63dff8-334a-44ad-b8a1-34bf5447b11f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:12.725965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:12.822184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee4a1d-f5e8-4270-8284-8be177768052 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:12.825053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63dff8-334a-44ad-b8a1-34bf5447b11f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:12.842668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:12.860745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b7a64-4310-403e-801b-2cce57eff893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:12.861759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b7a64-4310-403e-801b-2cce57eff893 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.862199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b7a64-4310-403e-801b-2cce57eff893 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.862613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b7a64-4310-403e-801b-2cce57eff893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.862801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1de159-1ccf-4977-ba74-2270d8ac71a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:12.863541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1de159-1ccf-4977-ba74-2270d8ac71a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.863819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1de159-1ccf-4977-ba74-2270d8ac71a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.864097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1de159-1ccf-4977-ba74-2270d8ac71a6 None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/b3ab5ed3d9e341f7853302ac7826879b/roles/67e4864efce9469bb6bdcca8ee5d5b24` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.874745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b7a64-4310-403e-801b-2cce57eff893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:12.876263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b7a64-4310-403e-801b-2cce57eff893 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:12.879347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1de159-1ccf-4977-ba74-2270d8ac71a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=b3ab5ed3d9e341f7853302ac7826879b, role_id=67e4864efce9469bb6bdcca8ee5d5b24)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:12.880759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1de159-1ccf-4977-ba74-2270d8ac71a6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:12.894215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 389/799] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:01:12 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/b3ab5ed3d9e341f7853302ac7826879b/roles/67e4864efce9469bb6bdcca8ee5d5b24 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-0e5b7a64-4310-403e-801b-2cce57eff893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:12.894472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:12.895420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:12.896421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:12.896421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:12.896421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:12.896421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:12.896421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 411/800] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:12.897209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cac880-d30c-4580-b1b4-7781622394b8 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.897501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cac880-d30c-4580-b1b4-7781622394b8 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.897786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cac880-d30c-4580-b1b4-7781622394b8 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.902852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 412/801] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:12.903011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1de159-1ccf-4977-ba74-2270d8ac71a6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:12.911068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd337a8a-0758-4500-a4a5-94ced5e1d7e8 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:12.911068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd337a8a-0758-4500-a4a5-94ced5e1d7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:12.911068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd337a8a-0758-4500-a4a5-94ced5e1d7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:12.929226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-dd337a8a-0758-4500-a4a5-94ced5e1d7e8 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:12.930541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-dd337a8a-0758-4500-a4a5-94ced5e1d7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:13.002242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:13.011094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 413/802] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:13.011094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd337a8a-0758-4500-a4a5-94ced5e1d7e8 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:13.051282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1de159-1ccf-4977-ba74-2270d8ac71a6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:13.095561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dad46a-9eec-4150-9069-e94073b9b18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:13.096464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dad46a-9eec-4150-9069-e94073b9b18b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.096746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dad46a-9eec-4150-9069-e94073b9b18b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.097017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dad46a-9eec-4150-9069-e94073b9b18b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.110405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dad46a-9eec-4150-9069-e94073b9b18b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:13.120984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd337a8a-0758-4500-a4a5-94ced5e1d7e8 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-15dad46a-9eec-4150-9069-e94073b9b18b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:13.121684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:13.122614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:13.123539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:13.124764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 390/803] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:13.129400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dad46a-9eec-4150-9069-e94073b9b18b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:13.164325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:13.169093 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7a986-1e27-4be4-8a2a-c5ebb80e9388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:13.170181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7a986-1e27-4be4-8a2a-c5ebb80e9388 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.170615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7a986-1e27-4be4-8a2a-c5ebb80e9388 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.171078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7a986-1e27-4be4-8a2a-c5ebb80e9388 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.172288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7a986-1e27-4be4-8a2a-c5ebb80e9388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:13.173527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7a986-1e27-4be4-8a2a-c5ebb80e9388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:13.175095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7a986-1e27-4be4-8a2a-c5ebb80e9388 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:13.189994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 414/804] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:13.195359 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7a986-1e27-4be4-8a2a-c5ebb80e9388 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:13.255525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dad46a-9eec-4150-9069-e94073b9b18b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:13.272629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:13.293257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c83dd-8a6d-4bd7-a3f4-2d33d2b790c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:13.294326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c83dd-8a6d-4bd7-a3f4-2d33d2b790c9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.294722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c83dd-8a6d-4bd7-a3f4-2d33d2b790c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.295150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c83dd-8a6d-4bd7-a3f4-2d33d2b790c9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.295924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c83dd-8a6d-4bd7-a3f4-2d33d2b790c9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:13.297440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c83dd-8a6d-4bd7-a3f4-2d33d2b790c9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-be9c83dd-8a6d-4bd7-a3f4-2d33d2b790c9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:13.310637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:13.311831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 05 15:01:13.312862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:13.320013 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 391/805] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Jul 5 15:01:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:13.321006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c83dd-8a6d-4bd7-a3f4-2d33d2b790c9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:13.330579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7a986-1e27-4be4-8a2a-c5ebb80e9388 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:13.380024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa56a2-a78a-4769-9a37-0626247668ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:13.381216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa56a2-a78a-4769-9a37-0626247668ea None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.381671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa56a2-a78a-4769-9a37-0626247668ea None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.382091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa56a2-a78a-4769-9a37-0626247668ea None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.383303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa56a2-a78a-4769-9a37-0626247668ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:13.384130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa56a2-a78a-4769-9a37-0626247668ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:13.386254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa56a2-a78a-4769-9a37-0626247668ea None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:13.404527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 415/806] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:01:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:13.405565 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa56a2-a78a-4769-9a37-0626247668ea None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:13.430797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:13.440756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c83dd-8a6d-4bd7-a3f4-2d33d2b790c9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:13.476220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f091f-c427-431b-9131-e219f0a4c753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:13.477019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f091f-c427-431b-9131-e219f0a4c753 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.477290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f091f-c427-431b-9131-e219f0a4c753 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.477570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f091f-c427-431b-9131-e219f0a4c753 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.486517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f091f-c427-431b-9131-e219f0a4c753 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:13.488014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f091f-c427-431b-9131-e219f0a4c753 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-985f091f-c427-431b-9131-e219f0a4c753 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:13.502291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:13.503303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 392/807] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 5 15:01:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:13.504261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f091f-c427-431b-9131-e219f0a4c753 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:13.515154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa56a2-a78a-4769-9a37-0626247668ea None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:13.540024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:13.555113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c4fa7-fada-4f75-adff-19f4cd3f3146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:13.556037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c4fa7-fada-4f75-adff-19f4cd3f3146 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.556531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c4fa7-fada-4f75-adff-19f4cd3f3146 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.556997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c4fa7-fada-4f75-adff-19f4cd3f3146 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.557778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c4fa7-fada-4f75-adff-19f4cd3f3146 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:13.558086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c4fa7-fada-4f75-adff-19f4cd3f3146 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:13.559496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8c4fa7-fada-4f75-adff-19f4cd3f3146 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:13.565062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 416/808] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 5 15:01:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:13.566551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83b1ed-fd3e-4768-9c25-96ff41f453ee None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.567201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83b1ed-fd3e-4768-9c25-96ff41f453ee None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.567687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e83b1ed-fd3e-4768-9c25-96ff41f453ee None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.573963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 417/809] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:13.573963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83b1ed-fd3e-4768-9c25-96ff41f453ee None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:13.611332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f091f-c427-431b-9131-e219f0a4c753 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:13.649268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec17e2-9011-44b4-add8-ee0638166b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:13.650644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec17e2-9011-44b4-add8-ee0638166b44 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.651227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec17e2-9011-44b4-add8-ee0638166b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.652287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec17e2-9011-44b4-add8-ee0638166b44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.665130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec17e2-9011-44b4-add8-ee0638166b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:13.666345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec17e2-9011-44b4-add8-ee0638166b44 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-f7ec17e2-9011-44b4-add8-ee0638166b44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:13.686579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:13.687680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:13.688572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:13.688572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:13.688572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:13.688572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:13.688572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 393/810] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:13.689558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec17e2-9011-44b4-add8-ee0638166b44 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:13.693799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:13.697295 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e83b1ed-fd3e-4768-9c25-96ff41f453ee None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:13.740546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5df537-5d2b-4b46-8574-d5c8ece51bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:13.741220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5df537-5d2b-4b46-8574-d5c8ece51bdb None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.741503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5df537-5d2b-4b46-8574-d5c8ece51bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.741780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5df537-5d2b-4b46-8574-d5c8ece51bdb None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.742622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5df537-5d2b-4b46-8574-d5c8ece51bdb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:13.742933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5df537-5d2b-4b46-8574-d5c8ece51bdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:13.744489 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5df537-5d2b-4b46-8574-d5c8ece51bdb None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:13.766192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 418/811] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 5 15:01:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:13.766192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e39cc-f8a4-4b14-9883-51f0b7e695e1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.766192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e39cc-f8a4-4b14-9883-51f0b7e695e1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.766192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7e39cc-f8a4-4b14-9883-51f0b7e695e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.780271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-cb7e39cc-f8a4-4b14-9883-51f0b7e695e1 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:13.781461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-cb7e39cc-f8a4-4b14-9883-51f0b7e695e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:13.803987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec17e2-9011-44b4-add8-ee0638166b44 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:13.817858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:13.841593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40279095-923b-46ca-badf-abd2b69ad191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:13.842382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40279095-923b-46ca-badf-abd2b69ad191 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:13.842671 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40279095-923b-46ca-badf-abd2b69ad191 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:13.842971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40279095-923b-46ca-badf-abd2b69ad191 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:13.843693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40279095-923b-46ca-badf-abd2b69ad191 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:13.844010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40279095-923b-46ca-badf-abd2b69ad191 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:13.845383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40279095-923b-46ca-badf-abd2b69ad191 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:13.850558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 394/812] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 5 15:01:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:13.851197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40279095-923b-46ca-badf-abd2b69ad191 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:13.857427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 419/813] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:13.858183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e39cc-f8a4-4b14-9883-51f0b7e695e1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:13.942008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:13.973590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7e39cc-f8a4-4b14-9883-51f0b7e695e1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:13.978941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40279095-923b-46ca-badf-abd2b69ad191 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:14.009544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efa2dd-8d02-4f59-bc98-ea9ff8ab994d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.010501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efa2dd-8d02-4f59-bc98-ea9ff8ab994d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.011666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efa2dd-8d02-4f59-bc98-ea9ff8ab994d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.012081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efa2dd-8d02-4f59-bc98-ea9ff8ab994d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.015748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ca290-2eed-44ef-9882-3ff745862d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.016479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ca290-2eed-44ef-9882-3ff745862d99 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.016710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ca290-2eed-44ef-9882-3ff745862d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.016997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ca290-2eed-44ef-9882-3ff745862d99 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.017733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ca290-2eed-44ef-9882-3ff745862d99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:14.018044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ca290-2eed-44ef-9882-3ff745862d99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.019456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ca290-2eed-44ef-9882-3ff745862d99 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.032808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 395/814] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:14.038659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ca290-2eed-44ef-9882-3ff745862d99 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:14.058629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:14.087513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60efa2dd-8d02-4f59-bc98-ea9ff8ab994d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.089343 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60efa2dd-8d02-4f59-bc98-ea9ff8ab994d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.152727 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ca290-2eed-44ef-9882-3ff745862d99 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:14.181304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:14.190159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478e73-38fc-42e5-9a8d-8cd31666abfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.195145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 420/815] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:01:14.195145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efa2dd-8d02-4f59-bc98-ea9ff8ab994d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:14.195370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478e73-38fc-42e5-9a8d-8cd31666abfa None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.195370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478e73-38fc-42e5-9a8d-8cd31666abfa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.195370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478e73-38fc-42e5-9a8d-8cd31666abfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.195909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478e73-38fc-42e5-9a8d-8cd31666abfa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.197429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478e73-38fc-42e5-9a8d-8cd31666abfa None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.224784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 396/816] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 5 15:01:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:14.225600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478e73-38fc-42e5-9a8d-8cd31666abfa None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:14.291480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:14.325584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efa2dd-8d02-4f59-bc98-ea9ff8ab994d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:14.376039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0e9ce-2087-48e5-a0b2-6802ece062b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.377539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0e9ce-2087-48e5-a0b2-6802ece062b3 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.377831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0e9ce-2087-48e5-a0b2-6802ece062b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.378115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0e9ce-2087-48e5-a0b2-6802ece062b3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.378797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0e9ce-2087-48e5-a0b2-6802ece062b3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.380860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0e9ce-2087-48e5-a0b2-6802ece062b3 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.388029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478e73-38fc-42e5-9a8d-8cd31666abfa None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-cfa0e9ce-2087-48e5-a0b2-6802ece062b3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:14.396784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:14.397864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:14.399349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 421/817] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:14 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:14.400157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0e9ce-2087-48e5-a0b2-6802ece062b3 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:14.437315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a73489-d6a8-4a59-95fe-f06768ffe27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.438579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a73489-d6a8-4a59-95fe-f06768ffe27d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.438579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a73489-d6a8-4a59-95fe-f06768ffe27d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.438796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a73489-d6a8-4a59-95fe-f06768ffe27d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.447190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a73489-d6a8-4a59-95fe-f06768ffe27d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.449165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a73489-d6a8-4a59-95fe-f06768ffe27d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c9a73489-d6a8-4a59-95fe-f06768ffe27d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:14.464425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:14.465428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 397/818] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:14.468601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a73489-d6a8-4a59-95fe-f06768ffe27d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:14.517636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:14.526024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0e9ce-2087-48e5-a0b2-6802ece062b3 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:14.563548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:14.565699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f644a54-b7d1-4895-b2ab-46595919d352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.566731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f644a54-b7d1-4895-b2ab-46595919d352 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.567026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f644a54-b7d1-4895-b2ab-46595919d352 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.567562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f644a54-b7d1-4895-b2ab-46595919d352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.577113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a73489-d6a8-4a59-95fe-f06768ffe27d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:14.580836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f644a54-b7d1-4895-b2ab-46595919d352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.582365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f644a54-b7d1-4895-b2ab-46595919d352 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-9f644a54-b7d1-4895-b2ab-46595919d352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:14.602165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:14.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:14.604495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:14.604495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:14.604495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:14.604495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:14.605427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 422/819] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:14.606270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f644a54-b7d1-4895-b2ab-46595919d352 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:14.620577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ecf74-b0d7-49ee-be1e-273e4636f3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.622353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ecf74-b0d7-49ee-be1e-273e4636f3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.622928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ecf74-b0d7-49ee-be1e-273e4636f3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.623437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ecf74-b0d7-49ee-be1e-273e4636f3f4 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.624537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ecf74-b0d7-49ee-be1e-273e4636f3f4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:14.624956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ecf74-b0d7-49ee-be1e-273e4636f3f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.626798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ecf74-b0d7-49ee-be1e-273e4636f3f4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.632574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 398/820] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:14.633599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ecf74-b0d7-49ee-be1e-273e4636f3f4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:14.716079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f644a54-b7d1-4895-b2ab-46595919d352 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:14.737444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ecf74-b0d7-49ee-be1e-273e4636f3f4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:14.747224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:14.754079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39de502-cc14-477e-a13f-058a03ab220c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.755480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39de502-cc14-477e-a13f-058a03ab220c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.756265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39de502-cc14-477e-a13f-058a03ab220c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.756615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39de502-cc14-477e-a13f-058a03ab220c None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.757601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39de502-cc14-477e-a13f-058a03ab220c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:14.757989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39de502-cc14-477e-a13f-058a03ab220c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.759666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39de502-cc14-477e-a13f-058a03ab220c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.773974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99a99e-0afe-4784-bc32-bf25c4f6dfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.774641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99a99e-0afe-4784-bc32-bf25c4f6dfee None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.774922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99a99e-0afe-4784-bc32-bf25c4f6dfee None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.775229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99a99e-0afe-4784-bc32-bf25c4f6dfee None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.776983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99a99e-0afe-4784-bc32-bf25c4f6dfee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:14.777307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99a99e-0afe-4784-bc32-bf25c4f6dfee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.778953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b99a99e-0afe-4784-bc32-bf25c4f6dfee None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.780576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 423/821] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 5 15:01:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:14.781586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39de502-cc14-477e-a13f-058a03ab220c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:14.803181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 399/822] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:14.804048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99a99e-0afe-4784-bc32-bf25c4f6dfee None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:14.844730 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:14.915995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39de502-cc14-477e-a13f-058a03ab220c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:14.921684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b99a99e-0afe-4784-bc32-bf25c4f6dfee None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:14.958891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c364261a-c6ff-40cd-9e36-1a5117f45b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.959730 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364261a-c6ff-40cd-9e36-1a5117f45b00 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.960006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364261a-c6ff-40cd-9e36-1a5117f45b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.960338 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c364261a-c6ff-40cd-9e36-1a5117f45b00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.965375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ef3fd-7255-492c-bb96-ed1e7491a264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:14.966214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ef3fd-7255-492c-bb96-ed1e7491a264 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.966644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ef3fd-7255-492c-bb96-ed1e7491a264 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.967107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ef3fd-7255-492c-bb96-ed1e7491a264 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.968470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ef3fd-7255-492c-bb96-ed1e7491a264 None admin] RBAC: Adding query filter params (type=network) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:14.968953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ef3fd-7255-492c-bb96-ed1e7491a264 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.971067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ef3fd-7255-492c-bb96-ed1e7491a264 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.972003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364261a-c6ff-40cd-9e36-1a5117f45b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:14.973442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c364261a-c6ff-40cd-9e36-1a5117f45b00 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:14.977578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 424/823] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:14 2022] GET /identity/v3/services?type=network => generated 350 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:14.978568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ef3fd-7255-492c-bb96-ed1e7491a264 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c364261a-c6ff-40cd-9e36-1a5117f45b00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:14.991756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:14.992714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:14.993615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:14.993615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:14.993615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:14.993615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:14.993615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 400/824] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:14.994569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c84b1-2cef-4f60-8e90-c9aef19566ae None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:14.994896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c84b1-2cef-4f60-8e90-c9aef19566ae None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:14.995200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c84b1-2cef-4f60-8e90-c9aef19566ae None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:14.999101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 401/825] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:14.999101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c84b1-2cef-4f60-8e90-c9aef19566ae None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:15.024403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:15.086604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ef3fd-7255-492c-bb96-ed1e7491a264 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:15.095428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:15.114539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c84b1-2cef-4f60-8e90-c9aef19566ae None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:15.124000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cf33d-6908-4af5-9b66-d61189a126fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:15.125014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cf33d-6908-4af5-9b66-d61189a126fc None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:15.125602 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cf33d-6908-4af5-9b66-d61189a126fc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:15.125884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3cf33d-6908-4af5-9b66-d61189a126fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:15.132861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cf33d-6908-4af5-9b66-d61189a126fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:15.134310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3cf33d-6908-4af5-9b66-d61189a126fc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-9e3cf33d-6908-4af5-9b66-d61189a126fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:15.148224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:15.149116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:15.150013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:15.150013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:15.150013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:15.150013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:15.150013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:15.150013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:15.150013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:15.150013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:15.151344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618fd91-0877-496c-9ccc-6aef2e26b9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:15.152008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618fd91-0877-496c-9ccc-6aef2e26b9b6 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:15.152295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618fd91-0877-496c-9ccc-6aef2e26b9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:15.152574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618fd91-0877-496c-9ccc-6aef2e26b9b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:15.153321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618fd91-0877-496c-9ccc-6aef2e26b9b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:15.154274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 425/826] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:15.154744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618fd91-0877-496c-9ccc-6aef2e26b9b6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:15.155120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cf33d-6908-4af5-9b66-d61189a126fc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:15.176430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 402/827] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 5 15:01:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:15.177721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d388a-67b6-49e2-b931-366dca6fb262 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:15.178015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d388a-67b6-49e2-b931-366dca6fb262 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:15.178294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d388a-67b6-49e2-b931-366dca6fb262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:15.199525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-db3d388a-67b6-49e2-b931-366dca6fb262 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:15.204082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-db3d388a-67b6-49e2-b931-366dca6fb262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:15.287317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 403/828] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:15.288495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d388a-67b6-49e2-b931-366dca6fb262 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:15.292613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:15.322055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3cf33d-6908-4af5-9b66-d61189a126fc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:15.373401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a158d8c-26b5-4f29-b2ec-7e654a584fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:15.373620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a158d8c-26b5-4f29-b2ec-7e654a584fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:15.375305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a158d8c-26b5-4f29-b2ec-7e654a584fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:15.375305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a158d8c-26b5-4f29-b2ec-7e654a584fc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:15.375869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a158d8c-26b5-4f29-b2ec-7e654a584fc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:15.376186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a158d8c-26b5-4f29-b2ec-7e654a584fc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:15.377581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a158d8c-26b5-4f29-b2ec-7e654a584fc8 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:15.396924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 426/829] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:15.397599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a158d8c-26b5-4f29-b2ec-7e654a584fc8 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:15.409691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d388a-67b6-49e2-b931-366dca6fb262 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:15.435315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:15.462878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a091ae4-1e64-4dfd-8dfc-672946bb70d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:15.464603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a091ae4-1e64-4dfd-8dfc-672946bb70d6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:15.464884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a091ae4-1e64-4dfd-8dfc-672946bb70d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:15.465159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a091ae4-1e64-4dfd-8dfc-672946bb70d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:15.465986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a091ae4-1e64-4dfd-8dfc-672946bb70d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:15.466301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a091ae4-1e64-4dfd-8dfc-672946bb70d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:15.467812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a091ae4-1e64-4dfd-8dfc-672946bb70d6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:15.481940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 404/830] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:15.482611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a091ae4-1e64-4dfd-8dfc-672946bb70d6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:15.548846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:15.590366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a091ae4-1e64-4dfd-8dfc-672946bb70d6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:15.625157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be75ad-e0b3-4f59-88eb-9204bd9049c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:15.625954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be75ad-e0b3-4f59-88eb-9204bd9049c7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:15.626242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be75ad-e0b3-4f59-88eb-9204bd9049c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:15.626525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be75ad-e0b3-4f59-88eb-9204bd9049c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:15.633264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be75ad-e0b3-4f59-88eb-9204bd9049c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:15.634699 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be75ad-e0b3-4f59-88eb-9204bd9049c7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:15.636747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a158d8c-26b5-4f29-b2ec-7e654a584fc8 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-27be75ad-e0b3-4f59-88eb-9204bd9049c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:15.648481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:15.649521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:15.650703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:15.650703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:15.650703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:15.650703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:15.650703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:15.650703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:15.650703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:15.650703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:15.650703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 405/831] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:15.650703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be75ad-e0b3-4f59-88eb-9204bd9049c7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:15.679512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd23d2a-9609-4cd8-9bcc-657329bba679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:15.680309 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd23d2a-9609-4cd8-9bcc-657329bba679 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:15.680590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd23d2a-9609-4cd8-9bcc-657329bba679 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:15.680868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd23d2a-9609-4cd8-9bcc-657329bba679 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:15.696277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd23d2a-9609-4cd8-9bcc-657329bba679 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-2cd23d2a-9609-4cd8-9bcc-657329bba679 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:15.705595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:15.706535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 427/832] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:01:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:15.708226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd23d2a-9609-4cd8-9bcc-657329bba679 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:15.765792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be75ad-e0b3-4f59-88eb-9204bd9049c7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:15.789929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:15.797488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:15.805180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f76d5-47a3-4175-9267-21f34cf2d945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:15.805843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f76d5-47a3-4175-9267-21f34cf2d945 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:15.806138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f76d5-47a3-4175-9267-21f34cf2d945 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:15.806418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f76d5-47a3-4175-9267-21f34cf2d945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:15.807300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f76d5-47a3-4175-9267-21f34cf2d945 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=e7a416f22d33450183801cef2b13822f) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:15.807659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f76d5-47a3-4175-9267-21f34cf2d945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:15.809113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f76d5-47a3-4175-9267-21f34cf2d945 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:15.815502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 406/833] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:15 2022] GET /identity/v3/role_assignments?user.id=e7a416f22d33450183801cef2b13822f&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:15.816177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f76d5-47a3-4175-9267-21f34cf2d945 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:15.824297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd23d2a-9609-4cd8-9bcc-657329bba679 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:15.873156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002fc2b-f73d-4cb6-b863-de57cee20bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:15.873877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002fc2b-f73d-4cb6-b863-de57cee20bbb None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:15.874160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002fc2b-f73d-4cb6-b863-de57cee20bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:15.874440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002fc2b-f73d-4cb6-b863-de57cee20bbb None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:15.875358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002fc2b-f73d-4cb6-b863-de57cee20bbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:15.875613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002fc2b-f73d-4cb6-b863-de57cee20bbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:15.877112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002fc2b-f73d-4cb6-b863-de57cee20bbb None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:15.895503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 428/834] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:15.897080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002fc2b-f73d-4cb6-b863-de57cee20bbb None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:15.952827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f76d5-47a3-4175-9267-21f34cf2d945 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:15.998637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc7b8f-e08f-4a7f-b083-d5e060a00c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:15.999960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc7b8f-e08f-4a7f-b083-d5e060a00c09 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:15.999960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc7b8f-e08f-4a7f-b083-d5e060a00c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:15.999960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc7b8f-e08f-4a7f-b083-d5e060a00c09 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.000624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc7b8f-e08f-4a7f-b083-d5e060a00c09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:16.001535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc7b8f-e08f-4a7f-b083-d5e060a00c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:16.002293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc7b8f-e08f-4a7f-b083-d5e060a00c09 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:16.013915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 407/835] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:01:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:16.019027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc7b8f-e08f-4a7f-b083-d5e060a00c09 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:16.021532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002fc2b-f73d-4cb6-b863-de57cee20bbb None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:16.066435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4e2e0-664e-4cc5-ba6d-3b5934c03552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:16.067379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4e2e0-664e-4cc5-ba6d-3b5934c03552 None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:16.067673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4e2e0-664e-4cc5-ba6d-3b5934c03552 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:16.067958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee4e2e0-664e-4cc5-ba6d-3b5934c03552 None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/4c73aa1f244d4251b8539c2806f38a42/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.069960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4e2e0-664e-4cc5-ba6d-3b5934c03552 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=4c73aa1f244d4251b8539c2806f38a42, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:16.071487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee4e2e0-664e-4cc5-ba6d-3b5934c03552 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:16.084349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:16.091816 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 429/836] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:01:15 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/4c73aa1f244d4251b8539c2806f38a42/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:01:16.144786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc7b8f-e08f-4a7f-b083-d5e060a00c09 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:16.190789 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082f04c-1327-48ff-8cbf-4bba2c499c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:16.191736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082f04c-1327-48ff-8cbf-4bba2c499c3d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:16.192023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082f04c-1327-48ff-8cbf-4bba2c499c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:16.192305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082f04c-1327-48ff-8cbf-4bba2c499c3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.205409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082f04c-1327-48ff-8cbf-4bba2c499c3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:16.206879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082f04c-1327-48ff-8cbf-4bba2c499c3d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4082f04c-1327-48ff-8cbf-4bba2c499c3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:16.226798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:16.227792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:16.228693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:16.228693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:16.228693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:16.228693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:16.228693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 408/837] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:16.234442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4e2e0-664e-4cc5-ba6d-3b5934c03552 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:16.248555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:16.349890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee4e2e0-664e-4cc5-ba6d-3b5934c03552 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:16.386304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab5305-d4ee-45f5-81e4-2e400f08b1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:16.387378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab5305-d4ee-45f5-81e4-2e400f08b1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:16.387889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab5305-d4ee-45f5-81e4-2e400f08b1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:16.388378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab5305-d4ee-45f5-81e4-2e400f08b1d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.389308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab5305-d4ee-45f5-81e4-2e400f08b1d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:16.389828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab5305-d4ee-45f5-81e4-2e400f08b1d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:16.391427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab5305-d4ee-45f5-81e4-2e400f08b1d2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:16.405108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 430/838] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:16.405443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9608c-52a9-4ef5-b35c-5517666f4c7d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:16.405800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9608c-52a9-4ef5-b35c-5517666f4c7d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:16.406095 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe9608c-52a9-4ef5-b35c-5517666f4c7d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.409265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 409/839] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:16.410358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab5305-d4ee-45f5-81e4-2e400f08b1d2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:16.412639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e9116-b5e2-4853-b0cc-e70574f21c23 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:16.412938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e9116-b5e2-4853-b0cc-e70574f21c23 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:16.413223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e9116-b5e2-4853-b0cc-e70574f21c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.430667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7e9e9116-b5e2-4853-b0cc-e70574f21c23 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:16.431925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7e9e9116-b5e2-4853-b0cc-e70574f21c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:16.487082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:16.507312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 410/840] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:16.525366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e9116-b5e2-4853-b0cc-e70574f21c23 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:16.527207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab5305-d4ee-45f5-81e4-2e400f08b1d2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:16.559762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:16.579883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bece0d-09eb-40e9-b272-1e867a5f738f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:16.580710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bece0d-09eb-40e9-b272-1e867a5f738f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:16.580999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bece0d-09eb-40e9-b272-1e867a5f738f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:16.581379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bece0d-09eb-40e9-b272-1e867a5f738f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.589413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bece0d-09eb-40e9-b272-1e867a5f738f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:16.590982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bece0d-09eb-40e9-b272-1e867a5f738f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-91bece0d-09eb-40e9-b272-1e867a5f738f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:16.607725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:16.608742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:16.609706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 431/841] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 5 15:01:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:16.614963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bece0d-09eb-40e9-b272-1e867a5f738f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:16.634033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e9116-b5e2-4853-b0cc-e70574f21c23 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:16.670064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b4db3-c29f-49d9-b7f7-b914401561bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:16.670790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b4db3-c29f-49d9-b7f7-b914401561bd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:16.671104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b4db3-c29f-49d9-b7f7-b914401561bd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:16.671391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b4db3-c29f-49d9-b7f7-b914401561bd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.672058 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b4db3-c29f-49d9-b7f7-b914401561bd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:16.673439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b4db3-c29f-49d9-b7f7-b914401561bd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-1b4b4db3-c29f-49d9-b7f7-b914401561bd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:16.685535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:16.687783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:01:16.689719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:16.699060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 411/842] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:01:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:16.699060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b4db3-c29f-49d9-b7f7-b914401561bd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:16.725497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bece0d-09eb-40e9-b272-1e867a5f738f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:16.753639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:16.765040 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce72536-2b7e-41aa-a9b9-38a896809cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:16.765929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce72536-2b7e-41aa-a9b9-38a896809cee None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:16.766364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce72536-2b7e-41aa-a9b9-38a896809cee None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:16.766790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce72536-2b7e-41aa-a9b9-38a896809cee None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.767876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce72536-2b7e-41aa-a9b9-38a896809cee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:16.768335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce72536-2b7e-41aa-a9b9-38a896809cee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:16.770143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce72536-2b7e-41aa-a9b9-38a896809cee None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:16.787738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:16.795185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b4db3-c29f-49d9-b7f7-b914401561bd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:16.795698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 432/843] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 5 15:01:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:16.798119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce72536-2b7e-41aa-a9b9-38a896809cee None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:16.831550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06254b4-ba87-4e53-adce-76bb0f736a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:16.832377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06254b4-ba87-4e53-adce-76bb0f736a04 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:16.832671 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06254b4-ba87-4e53-adce-76bb0f736a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:16.833258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06254b4-ba87-4e53-adce-76bb0f736a04 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.834258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06254b4-ba87-4e53-adce-76bb0f736a04 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:16.834573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06254b4-ba87-4e53-adce-76bb0f736a04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:16.835980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06254b4-ba87-4e53-adce-76bb0f736a04 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:16.843128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 412/844] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:01:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:16.847295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06254b4-ba87-4e53-adce-76bb0f736a04 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:16.912058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce72536-2b7e-41aa-a9b9-38a896809cee None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:16.952421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56b73a-20ed-4988-a4fa-a4895f86e5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:16.953913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56b73a-20ed-4988-a4fa-a4895f86e5df None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:16.954452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56b73a-20ed-4988-a4fa-a4895f86e5df None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:16.954879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56b73a-20ed-4988-a4fa-a4895f86e5df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:16.974743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06254b4-ba87-4e53-adce-76bb0f736a04 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:17.017884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47986e80-c597-49c1-a324-15462d58d135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:17.018679 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47986e80-c597-49c1-a324-15462d58d135 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:17.019464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47986e80-c597-49c1-a324-15462d58d135 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:17.019833 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47986e80-c597-49c1-a324-15462d58d135 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:17.020652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47986e80-c597-49c1-a324-15462d58d135 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:17.021040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47986e80-c597-49c1-a324-15462d58d135 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:17.022980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47986e80-c597-49c1-a324-15462d58d135 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:17.026926 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:17.031222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 413/845] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:01:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:17.052263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56b73a-20ed-4988-a4fa-a4895f86e5df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:17.054017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56b73a-20ed-4988-a4fa-a4895f86e5df None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:17.168417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 433/846] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:01:17.174154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47986e80-c597-49c1-a324-15462d58d135 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:17.212784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:17.297270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47986e80-c597-49c1-a324-15462d58d135 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:17.331851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef20abb-080e-47bb-8c21-ed8138cb1ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:17.332647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef20abb-080e-47bb-8c21-ed8138cb1ecd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:17.332932 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef20abb-080e-47bb-8c21-ed8138cb1ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:17.333213 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef20abb-080e-47bb-8c21-ed8138cb1ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:17.346913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef20abb-080e-47bb-8c21-ed8138cb1ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:17.348705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef20abb-080e-47bb-8c21-ed8138cb1ecd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-6ef20abb-080e-47bb-8c21-ed8138cb1ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:17.367239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:17.368169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:17.369094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:17.369094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:17.369094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:17.369094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:17.371887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 414/847] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:17.380118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56b73a-20ed-4988-a4fa-a4895f86e5df None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:17.465836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:17.482754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56b73a-20ed-4988-a4fa-a4895f86e5df None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:17.517466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6354ea7-2f53-4844-84ff-9becdf75e458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:17.518134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6354ea7-2f53-4844-84ff-9becdf75e458 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:17.518423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6354ea7-2f53-4844-84ff-9becdf75e458 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:17.518707 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6354ea7-2f53-4844-84ff-9becdf75e458 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:17.519567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6354ea7-2f53-4844-84ff-9becdf75e458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:17.519883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6354ea7-2f53-4844-84ff-9becdf75e458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:17.521254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6354ea7-2f53-4844-84ff-9becdf75e458 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:17.533284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 434/848] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:17.538841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef20abb-080e-47bb-8c21-ed8138cb1ecd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:17.622470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:17.653392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef20abb-080e-47bb-8c21-ed8138cb1ecd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:17.689380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953ca72-c0bc-4f78-93a9-bc1222a3b5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:17.690419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953ca72-c0bc-4f78-93a9-bc1222a3b5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:17.690897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953ca72-c0bc-4f78-93a9-bc1222a3b5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:17.691348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953ca72-c0bc-4f78-93a9-bc1222a3b5a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:17.698417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953ca72-c0bc-4f78-93a9-bc1222a3b5a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:17.700065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953ca72-c0bc-4f78-93a9-bc1222a3b5a3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4953ca72-c0bc-4f78-93a9-bc1222a3b5a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:17.721196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:17.722312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:17.723465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:17.723465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:17.723465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:17.723465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:17.723465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:17.723465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:17.723465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:17.723465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:17.725595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 415/849] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:17.732144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6354ea7-2f53-4844-84ff-9becdf75e458 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:17.836402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6354ea7-2f53-4844-84ff-9becdf75e458 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:17.872157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338954f4-c565-41b8-bd25-44adc7817dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:17.872827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338954f4-c565-41b8-bd25-44adc7817dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:17.873114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338954f4-c565-41b8-bd25-44adc7817dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:17.873393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338954f4-c565-41b8-bd25-44adc7817dc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:17.874192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338954f4-c565-41b8-bd25-44adc7817dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:17.874506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338954f4-c565-41b8-bd25-44adc7817dc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:17.875979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338954f4-c565-41b8-bd25-44adc7817dc1 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:17.888991 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 435/850] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:17.894852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953ca72-c0bc-4f78-93a9-bc1222a3b5a3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:17.971076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e217b87-91d8-4b65-b68c-6d4b847c4a17 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:17.971612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e217b87-91d8-4b65-b68c-6d4b847c4a17 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:17.971930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e217b87-91d8-4b65-b68c-6d4b847c4a17 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:17.974722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 436/851] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:17.978853 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a92db5-fcad-4d6f-b1c9-01163cdb8f8a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:17.979424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a92db5-fcad-4d6f-b1c9-01163cdb8f8a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:17.979829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a92db5-fcad-4d6f-b1c9-01163cdb8f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:17.994977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:17.997192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-01a92db5-fcad-4d6f-b1c9-01163cdb8f8a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:17.999218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-01a92db5-fcad-4d6f-b1c9-01163cdb8f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:18.003795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953ca72-c0bc-4f78-93a9-bc1222a3b5a3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:18.047462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5c295-c1fd-4c2f-9982-2b97da5c1256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:18.048126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5c295-c1fd-4c2f-9982-2b97da5c1256 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:18.048414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5c295-c1fd-4c2f-9982-2b97da5c1256 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:18.048696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f5c295-c1fd-4c2f-9982-2b97da5c1256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:18.049539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5c295-c1fd-4c2f-9982-2b97da5c1256 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=b3ab5ed3d9e341f7853302ac7826879b) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:18.049856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5c295-c1fd-4c2f-9982-2b97da5c1256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:18.051877 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f5c295-c1fd-4c2f-9982-2b97da5c1256 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:18.054421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:18.060993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 416/852] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:17 2022] GET /identity/v3/role_assignments?user.id=b3ab5ed3d9e341f7853302ac7826879b&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:18.095077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 437/853] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:18.112671 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5c295-c1fd-4c2f-9982-2b97da5c1256 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:18.201344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ae680-13cd-4fdd-a1c5-2845a4c5c20d None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:18.201634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ae680-13cd-4fdd-a1c5-2845a4c5c20d None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:18.201898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ae680-13cd-4fdd-a1c5-2845a4c5c20d None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:18.204678 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 438/854] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:18.209294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5748b90-d68a-47e6-82e6-0bba0e9cff00 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:18.209854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5748b90-d68a-47e6-82e6-0bba0e9cff00 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:18.210350 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5748b90-d68a-47e6-82e6-0bba0e9cff00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:18.228033 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-f5748b90-d68a-47e6-82e6-0bba0e9cff00 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:18.230121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f5748b90-d68a-47e6-82e6-0bba0e9cff00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:18.235363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f5c295-c1fd-4c2f-9982-2b97da5c1256 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:18.280940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b408bb-f771-446c-9b3f-72259cc8a7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:18.281748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b408bb-f771-446c-9b3f-72259cc8a7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:18.282046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b408bb-f771-446c-9b3f-72259cc8a7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:18.282308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b408bb-f771-446c-9b3f-72259cc8a7a0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:18.283621 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b408bb-f771-446c-9b3f-72259cc8a7a0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:18.285269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b408bb-f771-446c-9b3f-72259cc8a7a0 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-67b408bb-f771-446c-9b3f-72259cc8a7a0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:18.299976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:18.300960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:18.301807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 417/855] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Jul 5 15:01:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:18.308627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b408bb-f771-446c-9b3f-72259cc8a7a0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:18.327604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 439/856] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:18.352131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5748b90-d68a-47e6-82e6-0bba0e9cff00 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:18.389169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:18.421837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:18.441408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b408bb-f771-446c-9b3f-72259cc8a7a0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:18.460262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5748b90-d68a-47e6-82e6-0bba0e9cff00 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:18.490668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f8e13-c908-428a-b9f8-ac8b3dcac1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:18.492222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f8e13-c908-428a-b9f8-ac8b3dcac1ef None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:18.492660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f8e13-c908-428a-b9f8-ac8b3dcac1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:18.493343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f8e13-c908-428a-b9f8-ac8b3dcac1ef None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:18.494457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f8e13-c908-428a-b9f8-ac8b3dcac1ef None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:18.494944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f8e13-c908-428a-b9f8-ac8b3dcac1ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:18.497331 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f8e13-c908-428a-b9f8-ac8b3dcac1ef None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:18.500371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e2aba-5c02-45a9-872b-05cdf7e9ee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:18.501268 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e2aba-5c02-45a9-872b-05cdf7e9ee18 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:18.501652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e2aba-5c02-45a9-872b-05cdf7e9ee18 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:18.502028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e2aba-5c02-45a9-872b-05cdf7e9ee18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:18.503706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 418/857] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:18.512157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f8e13-c908-428a-b9f8-ac8b3dcac1ef None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:18.515453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e2aba-5c02-45a9-872b-05cdf7e9ee18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-8a2e2aba-5c02-45a9-872b-05cdf7e9ee18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:18.523404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:18.524342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:18.525177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 440/858] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:18.530898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e2aba-5c02-45a9-872b-05cdf7e9ee18 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:18.626999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f8e13-c908-428a-b9f8-ac8b3dcac1ef None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:18.652801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e2aba-5c02-45a9-872b-05cdf7e9ee18 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:18.661237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:18.673196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce580a2-260a-4e04-9939-e3d35640a26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:18.673870 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce580a2-260a-4e04-9939-e3d35640a26b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:18.674149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce580a2-260a-4e04-9939-e3d35640a26b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:18.674427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce580a2-260a-4e04-9939-e3d35640a26b None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:18.675721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce580a2-260a-4e04-9939-e3d35640a26b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:18.676035 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce580a2-260a-4e04-9939-e3d35640a26b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:18.678030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce580a2-260a-4e04-9939-e3d35640a26b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:18.678933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:18.686456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 419/859] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:18.702376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6f667-4591-4f85-947f-0e42ef55354e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:18.703115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6f667-4591-4f85-947f-0e42ef55354e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:18.703408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6f667-4591-4f85-947f-0e42ef55354e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:18.703687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6f667-4591-4f85-947f-0e42ef55354e None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:18.704476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6f667-4591-4f85-947f-0e42ef55354e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:18.704794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6f667-4591-4f85-947f-0e42ef55354e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:18.706294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6f667-4591-4f85-947f-0e42ef55354e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:18.716927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 441/860] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:01:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:18.722592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce580a2-260a-4e04-9939-e3d35640a26b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:18.831682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce580a2-260a-4e04-9939-e3d35640a26b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:18.873606 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328f6ec-8cc1-4ecc-83a2-361157a5515e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:18.874409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328f6ec-8cc1-4ecc-83a2-361157a5515e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:18.874697 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328f6ec-8cc1-4ecc-83a2-361157a5515e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:18.874975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d328f6ec-8cc1-4ecc-83a2-361157a5515e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:18.886410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328f6ec-8cc1-4ecc-83a2-361157a5515e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:18.887941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d328f6ec-8cc1-4ecc-83a2-361157a5515e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d328f6ec-8cc1-4ecc-83a2-361157a5515e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:18.906224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:18.907209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:18.908115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:18.908115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:18.908115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:18.908115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:18.910388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 420/861] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:18.916358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6f667-4591-4f85-947f-0e42ef55354e None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:19.026336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6f667-4591-4f85-947f-0e42ef55354e None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:19.061160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3a009-2162-4cb2-bcca-9d8ed0a05367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:19.061826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a009-2162-4cb2-bcca-9d8ed0a05367 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:19.062108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a009-2162-4cb2-bcca-9d8ed0a05367 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:19.062386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a009-2162-4cb2-bcca-9d8ed0a05367 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:19.063126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3a009-2162-4cb2-bcca-9d8ed0a05367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:19.063434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3a009-2162-4cb2-bcca-9d8ed0a05367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:19.064800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3a009-2162-4cb2-bcca-9d8ed0a05367 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:19.077833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 442/862] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:19.083525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328f6ec-8cc1-4ecc-83a2-361157a5515e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:19.130145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:19.189775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d328f6ec-8cc1-4ecc-83a2-361157a5515e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:19.225120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a676ff7-5440-40a2-b266-db7828ce67ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:19.225917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a676ff7-5440-40a2-b266-db7828ce67ae None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:19.226249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a676ff7-5440-40a2-b266-db7828ce67ae None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:19.226525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a676ff7-5440-40a2-b266-db7828ce67ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:19.234376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a676ff7-5440-40a2-b266-db7828ce67ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:19.235845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a676ff7-5440-40a2-b266-db7828ce67ae None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-6a676ff7-5440-40a2-b266-db7828ce67ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:19.250687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:19.251696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:19.252668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 421/863] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:19.257796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3a009-2162-4cb2-bcca-9d8ed0a05367 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:19.268322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fda21-da90-4cd8-92d6-bb87abaeea7c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:19.268612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fda21-da90-4cd8-92d6-bb87abaeea7c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:19.268933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fda21-da90-4cd8-92d6-bb87abaeea7c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:19.270536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 422/864] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:19.276376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88722c27-0ca5-4dcd-b894-b618360fc97c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:19.276721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88722c27-0ca5-4dcd-b894-b618360fc97c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:19.277002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88722c27-0ca5-4dcd-b894-b618360fc97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:19.295157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-88722c27-0ca5-4dcd-b894-b618360fc97c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:19.296348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-88722c27-0ca5-4dcd-b894-b618360fc97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:19.321020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:19.377137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 423/865] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:19.392089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3a009-2162-4cb2-bcca-9d8ed0a05367 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:19.398514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88722c27-0ca5-4dcd-b894-b618360fc97c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:19.432040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:19.438650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d32bb3-2f1e-4059-8e4f-71ecbe1ecc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:19.439365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d32bb3-2f1e-4059-8e4f-71ecbe1ecc0d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:19.439650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d32bb3-2f1e-4059-8e4f-71ecbe1ecc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:19.439924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d32bb3-2f1e-4059-8e4f-71ecbe1ecc0d None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:19.440765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d32bb3-2f1e-4059-8e4f-71ecbe1ecc0d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:19.441075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d32bb3-2f1e-4059-8e4f-71ecbe1ecc0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:19.442512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d32bb3-2f1e-4059-8e4f-71ecbe1ecc0d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:19.467255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 443/866] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 5 15:01:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:19.472773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d32bb3-2f1e-4059-8e4f-71ecbe1ecc0d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:19.516372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88722c27-0ca5-4dcd-b894-b618360fc97c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:19.553580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ca640-bc33-4e54-87ac-09e74709899f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:19.554372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ca640-bc33-4e54-87ac-09e74709899f None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:19.554666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ca640-bc33-4e54-87ac-09e74709899f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:19.554943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ca640-bc33-4e54-87ac-09e74709899f None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:19.555693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ca640-bc33-4e54-87ac-09e74709899f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:19.557223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ca640-bc33-4e54-87ac-09e74709899f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:19.578971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 424/867] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:01:19 2022] POST /identity/v3/services => generated 251 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:19.604845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d32bb3-2f1e-4059-8e4f-71ecbe1ecc0d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:19.607887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:19.652252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f991e-0b34-49ac-bd72-e138eca34b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:19.652829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f991e-0b34-49ac-bd72-e138eca34b0c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:19.653112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f991e-0b34-49ac-bd72-e138eca34b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:19.653389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f991e-0b34-49ac-bd72-e138eca34b0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:19.731442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f991e-0b34-49ac-bd72-e138eca34b0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:19.732911 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f991e-0b34-49ac-bd72-e138eca34b0c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:19.830704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:19.833684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 444/868] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:19 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:01:19.839527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ca640-bc33-4e54-87ac-09e74709899f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:19.939098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:19.949006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ca640-bc33-4e54-87ac-09e74709899f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:19.985753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2352ad0-206e-4c82-92f0-2e24eb462b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:19.986549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2352ad0-206e-4c82-92f0-2e24eb462b8a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:19.986834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2352ad0-206e-4c82-92f0-2e24eb462b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:19.987141 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2352ad0-206e-4c82-92f0-2e24eb462b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:19.998118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2352ad0-206e-4c82-92f0-2e24eb462b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:19.999609 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2352ad0-206e-4c82-92f0-2e24eb462b8a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-a2352ad0-206e-4c82-92f0-2e24eb462b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:20.017950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:20.018892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:20.020254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:20.020254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:20.020254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:20.020254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:20.020254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 425/869] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:20.028126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f991e-0b34-49ac-bd72-e138eca34b0c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:20.130021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f991e-0b34-49ac-bd72-e138eca34b0c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:20.170768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1cc25e-1836-4b9d-b9f7-a76a00d0a1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:20.171766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cc25e-1836-4b9d-b9f7-a76a00d0a1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:20.172234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cc25e-1836-4b9d-b9f7-a76a00d0a1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:20.172689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1cc25e-1836-4b9d-b9f7-a76a00d0a1b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:20.173625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cc25e-1836-4b9d-b9f7-a76a00d0a1b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:20.174123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cc25e-1836-4b9d-b9f7-a76a00d0a1b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:20.175700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1cc25e-1836-4b9d-b9f7-a76a00d0a1b0 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:20.193504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2352ad0-206e-4c82-92f0-2e24eb462b8a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:20.206274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 445/870] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:20.296818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2352ad0-206e-4c82-92f0-2e24eb462b8a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:20.333770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2124f-30b7-466a-8356-05a3aab2954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:20.334580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2124f-30b7-466a-8356-05a3aab2954b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:20.335161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2124f-30b7-466a-8356-05a3aab2954b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:20.335445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2124f-30b7-466a-8356-05a3aab2954b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:20.342406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff198ce7-bc02-4d90-9b7d-621ee8715630 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:20.342936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2124f-30b7-466a-8356-05a3aab2954b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:20.343143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff198ce7-bc02-4d90-9b7d-621ee8715630 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:20.343591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff198ce7-bc02-4d90-9b7d-621ee8715630 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:20.344421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2124f-30b7-466a-8356-05a3aab2954b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:20.345597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 446/871] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:20.350706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664514-1461-4dfd-ae90-27c197645eca None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:20.351315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664514-1461-4dfd-ae90-27c197645eca None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:20.351800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48664514-1461-4dfd-ae90-27c197645eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-43a2124f-30b7-466a-8356-05a3aab2954b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:20.360117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:20.361044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:20.361944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:20.361944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:20.361944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:20.361944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:20.361944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:20.361944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:20.361944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:20.361944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:20.361944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 426/872] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:20.367137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2124f-30b7-466a-8356-05a3aab2954b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:20.372879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:20.376113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-48664514-1461-4dfd-ae90-27c197645eca None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:20.377737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-48664514-1461-4dfd-ae90-27c197645eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:20.453249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:20.453684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 447/873] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:20.474321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48664514-1461-4dfd-ae90-27c197645eca None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:20.486216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2124f-30b7-466a-8356-05a3aab2954b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:20.523668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dafd6-0e9a-4c72-b9ad-1e0a84bcec41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:20.524695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dafd6-0e9a-4c72-b9ad-1e0a84bcec41 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:20.525187 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dafd6-0e9a-4c72-b9ad-1e0a84bcec41 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:20.525689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dafd6-0e9a-4c72-b9ad-1e0a84bcec41 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:20.526711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dafd6-0e9a-4c72-b9ad-1e0a84bcec41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:20.527361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dafd6-0e9a-4c72-b9ad-1e0a84bcec41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:20.529001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dafd6-0e9a-4c72-b9ad-1e0a84bcec41 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:20.547703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 427/874] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:20.549114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dafd6-0e9a-4c72-b9ad-1e0a84bcec41 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:20.605364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48664514-1461-4dfd-ae90-27c197645eca None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:20.617522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:20.646244 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc6072-723a-43f4-ac64-3acc270ef3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:20.647468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc6072-723a-43f4-ac64-3acc270ef3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:20.647984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc6072-723a-43f4-ac64-3acc270ef3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:20.648471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc6072-723a-43f4-ac64-3acc270ef3d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:20.659867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc6072-723a-43f4-ac64-3acc270ef3d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:20.665111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dafd6-0e9a-4c72-b9ad-1e0a84bcec41 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-e1cc6072-723a-43f4-ac64-3acc270ef3d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:20.668703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:20.669662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:20.670507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:20.677958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 448/875] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 5 15:01:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:20.679137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc6072-723a-43f4-ac64-3acc270ef3d7 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:20.706462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f413a-a4c4-415a-8295-4157360fe804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:20.707195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f413a-a4c4-415a-8295-4157360fe804 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:20.707482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f413a-a4c4-415a-8295-4157360fe804 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:20.707845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f413a-a4c4-415a-8295-4157360fe804 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:20.708699 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f413a-a4c4-415a-8295-4157360fe804 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=4c73aa1f244d4251b8539c2806f38a42) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:20.709014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f413a-a4c4-415a-8295-4157360fe804 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:20.710681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f413a-a4c4-415a-8295-4157360fe804 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:20.718352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 428/876] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:20 2022] GET /identity/v3/role_assignments?user.id=4c73aa1f244d4251b8539c2806f38a42&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:20.734817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:20.793752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc6072-723a-43f4-ac64-3acc270ef3d7 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:20.817868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e3b85-d1a3-4dc8-b574-e1b9033dff0d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:20.818157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e3b85-d1a3-4dc8-b574-e1b9033dff0d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:20.818440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e3b85-d1a3-4dc8-b574-e1b9033dff0d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:20.819904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 429/877] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:20.826266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a131baf-4c47-46ff-8a31-ac81c4b0bbb7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:20.826881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a131baf-4c47-46ff-8a31-ac81c4b0bbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:20.827403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a131baf-4c47-46ff-8a31-ac81c4b0bbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:20.832137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bc0f0-1c94-4169-bb59-b9f33506e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:20.832824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc0f0-1c94-4169-bb59-b9f33506e416 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:20.833111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc0f0-1c94-4169-bb59-b9f33506e416 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:20.833392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bc0f0-1c94-4169-bb59-b9f33506e416 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:20.834153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc0f0-1c94-4169-bb59-b9f33506e416 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:20.834466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc0f0-1c94-4169-bb59-b9f33506e416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:20.836320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bc0f0-1c94-4169-bb59-b9f33506e416 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:20.845885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 449/878] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 5 15:01:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:20.849552 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1a131baf-4c47-46ff-8a31-ac81c4b0bbb7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:20.851180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bc0f0-1c94-4169-bb59-b9f33506e416 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:20.851876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1a131baf-4c47-46ff-8a31-ac81c4b0bbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:20.931520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 430/879] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:20.955009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a131baf-4c47-46ff-8a31-ac81c4b0bbb7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:20.968974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bc0f0-1c94-4169-bb59-b9f33506e416 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:20.984995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:21.008015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac9c78-24d9-4be3-81db-ac95efbdc417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:21.009153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac9c78-24d9-4be3-81db-ac95efbdc417 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:21.010049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac9c78-24d9-4be3-81db-ac95efbdc417 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:21.011111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac9c78-24d9-4be3-81db-ac95efbdc417 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:21.024917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac9c78-24d9-4be3-81db-ac95efbdc417 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:21.027156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac9c78-24d9-4be3-81db-ac95efbdc417 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:21.030099 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-bfac9c78-24d9-4be3-81db-ac95efbdc417 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:21.049787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:21.051067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:21.051975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:21.051975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:21.051975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:21.051975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:21.052965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 450/880] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:21.058278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac9c78-24d9-4be3-81db-ac95efbdc417 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:21.081320 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a131baf-4c47-46ff-8a31-ac81c4b0bbb7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:21.126648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b02b48-c6a8-44fa-8f87-db1cb9cc2147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:21.128323 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b02b48-c6a8-44fa-8f87-db1cb9cc2147 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:21.128947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b02b48-c6a8-44fa-8f87-db1cb9cc2147 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:21.129522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b02b48-c6a8-44fa-8f87-db1cb9cc2147 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:21.130427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b02b48-c6a8-44fa-8f87-db1cb9cc2147 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:21.132265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b02b48-c6a8-44fa-8f87-db1cb9cc2147 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:21.154652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 431/881] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:01:20 2022] POST /identity/v3/services => generated 269 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:01:21.175995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac9c78-24d9-4be3-81db-ac95efbdc417 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:21.222830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16449d-10a4-4cb1-99af-bed3d904971c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:21.223459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16449d-10a4-4cb1-99af-bed3d904971c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:21.223744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16449d-10a4-4cb1-99af-bed3d904971c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:21.224019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16449d-10a4-4cb1-99af-bed3d904971c None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:21.224761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16449d-10a4-4cb1-99af-bed3d904971c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:21.225069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16449d-10a4-4cb1-99af-bed3d904971c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:21.226552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16449d-10a4-4cb1-99af-bed3d904971c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:21.240984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 451/882] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:21.252217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b02b48-c6a8-44fa-8f87-db1cb9cc2147 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:21.364813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b02b48-c6a8-44fa-8f87-db1cb9cc2147 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:21.382884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:21.403110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33da85-b2b8-48c4-956c-8f8515a887a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:21.403886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33da85-b2b8-48c4-956c-8f8515a887a9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:21.404177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33da85-b2b8-48c4-956c-8f8515a887a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:21.404456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33da85-b2b8-48c4-956c-8f8515a887a9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:21.412495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33da85-b2b8-48c4-956c-8f8515a887a9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:21.413949 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33da85-b2b8-48c4-956c-8f8515a887a9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-fb33da85-b2b8-48c4-956c-8f8515a887a9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:21.429014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:21.430010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:21.430965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 432/883] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 5 15:01:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:21.437755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16449d-10a4-4cb1-99af-bed3d904971c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:21.461426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:21.545286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16449d-10a4-4cb1-99af-bed3d904971c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:21.581698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774060ef-709e-4455-8258-d0be3f9aba38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:21.582436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774060ef-709e-4455-8258-d0be3f9aba38 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:21.582729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774060ef-709e-4455-8258-d0be3f9aba38 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:21.583010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774060ef-709e-4455-8258-d0be3f9aba38 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:21.583954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774060ef-709e-4455-8258-d0be3f9aba38 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:21.584273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774060ef-709e-4455-8258-d0be3f9aba38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:21.585728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774060ef-709e-4455-8258-d0be3f9aba38 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:21.604622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 452/884] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 5 15:01:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:21.610109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33da85-b2b8-48c4-956c-8f8515a887a9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:21.705236 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ddbf-56c8-487f-a400-97c3f587dd44 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:21.705881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ddbf-56c8-487f-a400-97c3f587dd44 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:21.706364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ddbf-56c8-487f-a400-97c3f587dd44 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:21.709429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 453/885] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:21.714187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999c60-764c-4f95-b5c3-823e42e99b2e None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:21.714483 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999c60-764c-4f95-b5c3-823e42e99b2e None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:21.714765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49999c60-764c-4f95-b5c3-823e42e99b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:21.721728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33da85-b2b8-48c4-956c-8f8515a887a9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:21.735555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-49999c60-764c-4f95-b5c3-823e42e99b2e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:21.737862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-49999c60-764c-4f95-b5c3-823e42e99b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:21.757363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18d9d5-af51-4442-88e0-20f1f40e0883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:21.758049 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18d9d5-af51-4442-88e0-20f1f40e0883 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:21.758338 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18d9d5-af51-4442-88e0-20f1f40e0883 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:21.758607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18d9d5-af51-4442-88e0-20f1f40e0883 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:21.811794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:21.841326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 454/886] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:21.843914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18d9d5-af51-4442-88e0-20f1f40e0883 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:21.845636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18d9d5-af51-4442-88e0-20f1f40e0883 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:21.859957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49999c60-764c-4f95-b5c3-823e42e99b2e None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:21.871132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:21.984071 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49999c60-764c-4f95-b5c3-823e42e99b2e None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:21.985653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 433/887] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:21 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:01:21.986571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18d9d5-af51-4442-88e0-20f1f40e0883 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:22.022506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646a13b-2203-4cbc-a43f-3e5a9649ddb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:22.023806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646a13b-2203-4cbc-a43f-3e5a9649ddb6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.024296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646a13b-2203-4cbc-a43f-3e5a9649ddb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.024849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646a13b-2203-4cbc-a43f-3e5a9649ddb6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.025705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646a13b-2203-4cbc-a43f-3e5a9649ddb6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:22.027341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646a13b-2203-4cbc-a43f-3e5a9649ddb6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-3646a13b-2203-4cbc-a43f-3e5a9649ddb6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:22.040058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:22.041048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:01:22.042446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:22.050138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 455/888] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:01:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:22.051780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646a13b-2203-4cbc-a43f-3e5a9649ddb6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:22.071433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:22.095346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18d9d5-af51-4442-88e0-20f1f40e0883 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:22.128528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:22.133496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54d6a6-f862-4614-84db-5cd1540a89d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:22.134295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54d6a6-f862-4614-84db-5cd1540a89d7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.134618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54d6a6-f862-4614-84db-5cd1540a89d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.134897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd54d6a6-f862-4614-84db-5cd1540a89d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.146594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54d6a6-f862-4614-84db-5cd1540a89d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:22.148177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd54d6a6-f862-4614-84db-5cd1540a89d7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:22.162642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646a13b-2203-4cbc-a43f-3e5a9649ddb6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-dd54d6a6-f862-4614-84db-5cd1540a89d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:22.167944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:22.169639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:22.170775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:22.170775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:22.170775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:22.170775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:22.173083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 434/889] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:22.173943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54d6a6-f862-4614-84db-5cd1540a89d7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:22.205898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213d561-469e-4f46-9e2e-f52c6401b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:22.206898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213d561-469e-4f46-9e2e-f52c6401b31f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.207437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213d561-469e-4f46-9e2e-f52c6401b31f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.207920 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213d561-469e-4f46-9e2e-f52c6401b31f None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.208834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213d561-469e-4f46-9e2e-f52c6401b31f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:22.209411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213d561-469e-4f46-9e2e-f52c6401b31f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:22.210976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213d561-469e-4f46-9e2e-f52c6401b31f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:22.221947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 456/890] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:01:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:22.223314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213d561-469e-4f46-9e2e-f52c6401b31f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:22.294195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd54d6a6-f862-4614-84db-5cd1540a89d7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:22.326176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:22.330299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213d561-469e-4f46-9e2e-f52c6401b31f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:22.336198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7be4d5-21c1-4d14-966f-cb1f7a03922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:22.337048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7be4d5-21c1-4d14-966f-cb1f7a03922a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.337482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7be4d5-21c1-4d14-966f-cb1f7a03922a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.337935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7be4d5-21c1-4d14-966f-cb1f7a03922a None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.339906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7be4d5-21c1-4d14-966f-cb1f7a03922a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:22.340374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7be4d5-21c1-4d14-966f-cb1f7a03922a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:22.342351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7be4d5-21c1-4d14-966f-cb1f7a03922a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:22.360669 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 435/891] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:22.366452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7be4d5-21c1-4d14-966f-cb1f7a03922a None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:22.377939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf7971-a9d5-4f88-8c64-f35d0ce3d44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:22.378711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf7971-a9d5-4f88-8c64-f35d0ce3d44c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.379090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf7971-a9d5-4f88-8c64-f35d0ce3d44c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.379459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf7971-a9d5-4f88-8c64-f35d0ce3d44c None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.380275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf7971-a9d5-4f88-8c64-f35d0ce3d44c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:22.380662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf7971-a9d5-4f88-8c64-f35d0ce3d44c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:22.382444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf7971-a9d5-4f88-8c64-f35d0ce3d44c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:22.384899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:22.393576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 457/892] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:01:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:22.394620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf7971-a9d5-4f88-8c64-f35d0ce3d44c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:22.493937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7be4d5-21c1-4d14-966f-cb1f7a03922a None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:22.515994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf7971-a9d5-4f88-8c64-f35d0ce3d44c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:22.539434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdacdea-9c44-4195-b89a-baa99a67cf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:22.540769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdacdea-9c44-4195-b89a-baa99a67cf5f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.541239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdacdea-9c44-4195-b89a-baa99a67cf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.542475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdacdea-9c44-4195-b89a-baa99a67cf5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.552435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdacdea-9c44-4195-b89a-baa99a67cf5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:22.553888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdacdea-9c44-4195-b89a-baa99a67cf5f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:22.562232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e6656-1357-495c-ab7a-fab36b260f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:22.563209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e6656-1357-495c-ab7a-fab36b260f62 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.563730 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e6656-1357-495c-ab7a-fab36b260f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.564229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759e6656-1357-495c-ab7a-fab36b260f62 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.565226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e6656-1357-495c-ab7a-fab36b260f62 None admin] RBAC: Adding query filter params (interface=public, service_id=7690baa40a644b66a82923e847f79b11, region_id=RegionOne) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:22.565743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e6656-1357-495c-ab7a-fab36b260f62 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:22.567342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759e6656-1357-495c-ab7a-fab36b260f62 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-cbdacdea-9c44-4195-b89a-baa99a67cf5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:22.568590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:22.569444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:22.570341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:22.570341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:22.570341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:22.570341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:22.570341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:22.570341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:22.570341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:22.570341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:22.575083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 436/893] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:22.576447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdacdea-9c44-4195-b89a-baa99a67cf5f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:22.580927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 458/894] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 5 15:01:22 2022] GET /identity/v3/endpoints?service_id=7690baa40a644b66a82923e847f79b11&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:22.586706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e6656-1357-495c-ab7a-fab36b260f62 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:22.633106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:22.674855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:22.686475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdacdea-9c44-4195-b89a-baa99a67cf5f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:22.718677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759e6656-1357-495c-ab7a-fab36b260f62 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:22.723064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12244c-3431-4346-8e2e-d9b246551879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:22.724016 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12244c-3431-4346-8e2e-d9b246551879 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.724526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12244c-3431-4346-8e2e-d9b246551879 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.725001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12244c-3431-4346-8e2e-d9b246551879 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.726038 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12244c-3431-4346-8e2e-d9b246551879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:22.726533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12244c-3431-4346-8e2e-d9b246551879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:22.728199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12244c-3431-4346-8e2e-d9b246551879 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:22.745912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 437/895] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:22.747099 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12244c-3431-4346-8e2e-d9b246551879 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:22.756936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625799c-57d8-4de0-aa14-755cbecc2de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:22.757687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625799c-57d8-4de0-aa14-755cbecc2de7 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.758041 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625799c-57d8-4de0-aa14-755cbecc2de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.758390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625799c-57d8-4de0-aa14-755cbecc2de7 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.759533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625799c-57d8-4de0-aa14-755cbecc2de7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:22.761251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625799c-57d8-4de0-aa14-755cbecc2de7 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:22.768159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 459/896] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 5 15:01:22 2022] GET /identity/v3/services => generated 1268 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:01:22.862916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12244c-3431-4346-8e2e-d9b246551879 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:22.875081 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:22.880902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e57e9b-2e94-4b54-97c8-be10f26de99c None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.881199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e57e9b-2e94-4b54-97c8-be10f26de99c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.881484 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e57e9b-2e94-4b54-97c8-be10f26de99c None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.882860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 460/897] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:22.891560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa542f-5d14-4b04-b21b-f27276e7ed11 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.891934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa542f-5d14-4b04-b21b-f27276e7ed11 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.892291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa542f-5d14-4b04-b21b-f27276e7ed11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.903664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaada3-105f-4e05-9289-5ea9ff9d23b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:22.904685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaada3-105f-4e05-9289-5ea9ff9d23b1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:22.905153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaada3-105f-4e05-9289-5ea9ff9d23b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:22.905628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaada3-105f-4e05-9289-5ea9ff9d23b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:22.906672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaada3-105f-4e05-9289-5ea9ff9d23b1 None admin] RBAC: Adding query filter params (role.id=e08bb5ad422e4f6593bcbaca2bdf4366, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=b3ab5ed3d9e341f7853302ac7826879b) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:22.907244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaada3-105f-4e05-9289-5ea9ff9d23b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:22.909029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaada3-105f-4e05-9289-5ea9ff9d23b1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:22.911779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-28aa542f-5d14-4b04-b21b-f27276e7ed11 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:22.913240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-28aa542f-5d14-4b04-b21b-f27276e7ed11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:22.916727 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 438/898] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:22 2022] GET /identity/v3/role_assignments?user.id=b3ab5ed3d9e341f7853302ac7826879b&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=e08bb5ad422e4f6593bcbaca2bdf4366 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:22.995077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 461/899] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:22 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:23.019329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaada3-105f-4e05-9289-5ea9ff9d23b1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:23.129502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaada3-105f-4e05-9289-5ea9ff9d23b1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:23.164649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575bdae-f49f-4460-8e1e-16e0895e9529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:23.165862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575bdae-f49f-4460-8e1e-16e0895e9529 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:23.166460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575bdae-f49f-4460-8e1e-16e0895e9529 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:23.166948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575bdae-f49f-4460-8e1e-16e0895e9529 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:23.167839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575bdae-f49f-4460-8e1e-16e0895e9529 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:23.169427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0575bdae-f49f-4460-8e1e-16e0895e9529 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0575bdae-f49f-4460-8e1e-16e0895e9529 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:23.182498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:23.183551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:23.184400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:23.187275 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 439/900] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:23.193918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa542f-5d14-4b04-b21b-f27276e7ed11 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:23.259483 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:23.302925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa542f-5d14-4b04-b21b-f27276e7ed11 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:23.306412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78086a9b-c736-468d-9068-d12237855590 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:23.306708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78086a9b-c736-468d-9068-d12237855590 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:23.307106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78086a9b-c736-468d-9068-d12237855590 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:23.311156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 440/901] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:23.314287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea05734-543d-47af-adb5-e0d29443a0d9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:23.314581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea05734-543d-47af-adb5-e0d29443a0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:23.314867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea05734-543d-47af-adb5-e0d29443a0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:23.332224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-aea05734-543d-47af-adb5-e0d29443a0d9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:23.333433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-aea05734-543d-47af-adb5-e0d29443a0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:23.345921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c556af-df68-4863-ad8f-2d14d8358c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:23.347167 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c556af-df68-4863-ad8f-2d14d8358c3c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:23.347661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c556af-df68-4863-ad8f-2d14d8358c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:23.348377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c556af-df68-4863-ad8f-2d14d8358c3c None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:23.349758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c556af-df68-4863-ad8f-2d14d8358c3c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:23.350393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c556af-df68-4863-ad8f-2d14d8358c3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:23.352359 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c556af-df68-4863-ad8f-2d14d8358c3c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:23.359754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 462/902] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 5 15:01:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:23.364956 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c556af-df68-4863-ad8f-2d14d8358c3c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:23.367935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:23.415265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 441/903] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:23.434266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea05734-543d-47af-adb5-e0d29443a0d9 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:23.473314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c556af-df68-4863-ad8f-2d14d8358c3c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:23.508254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d2a5b-bae0-4d0e-88ee-473f1b772445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:23.509724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d2a5b-bae0-4d0e-88ee-473f1b772445 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:23.509724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d2a5b-bae0-4d0e-88ee-473f1b772445 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:23.509923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d2a5b-bae0-4d0e-88ee-473f1b772445 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:23.511286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d2a5b-bae0-4d0e-88ee-473f1b772445 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:23.511699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d2a5b-bae0-4d0e-88ee-473f1b772445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:23.513183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d2a5b-bae0-4d0e-88ee-473f1b772445 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:23.518331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 463/904] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 5 15:01:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:23.519370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:23.545397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea05734-543d-47af-adb5-e0d29443a0d9 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:23.586502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b82d66-7320-4d06-9831-951502fa916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:23.587418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b82d66-7320-4d06-9831-951502fa916b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:23.587710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b82d66-7320-4d06-9831-951502fa916b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:23.587990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b82d66-7320-4d06-9831-951502fa916b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:23.588744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b82d66-7320-4d06-9831-951502fa916b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:23.590162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b82d66-7320-4d06-9831-951502fa916b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-b1b82d66-7320-4d06-9831-951502fa916b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:23.603607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:23.605161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:01:23.606011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:23.610535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 442/905] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Jul 5 15:01:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:23.612051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d2a5b-bae0-4d0e-88ee-473f1b772445 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:23.730791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d2a5b-bae0-4d0e-88ee-473f1b772445 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:23.767752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64e971-b577-4cad-9d85-a85218d6a1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:23.768429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64e971-b577-4cad-9d85-a85218d6a1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:23.768719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64e971-b577-4cad-9d85-a85218d6a1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:23.769004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64e971-b577-4cad-9d85-a85218d6a1a1 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:23.769721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64e971-b577-4cad-9d85-a85218d6a1a1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:23.770035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64e971-b577-4cad-9d85-a85218d6a1a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:23.771447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64e971-b577-4cad-9d85-a85218d6a1a1 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:23.777399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 464/906] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:23.782864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b82d66-7320-4d06-9831-951502fa916b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:23.819418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:23.893738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b82d66-7320-4d06-9831-951502fa916b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:23.931272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a17a52-2f3b-4bd3-b8e3-28887c0cf2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:23.932504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a17a52-2f3b-4bd3-b8e3-28887c0cf2b1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:23.932794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a17a52-2f3b-4bd3-b8e3-28887c0cf2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:23.933077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a17a52-2f3b-4bd3-b8e3-28887c0cf2b1 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:23.933870 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a17a52-2f3b-4bd3-b8e3-28887c0cf2b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:23.934193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a17a52-2f3b-4bd3-b8e3-28887c0cf2b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:23.935618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a17a52-2f3b-4bd3-b8e3-28887c0cf2b1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:23.946085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 443/907] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:23.947500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64e971-b577-4cad-9d85-a85218d6a1a1 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:24.000535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:24.056598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64e971-b577-4cad-9d85-a85218d6a1a1 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:24.094950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40f142b-ea5d-430f-bf0a-61f1da74df4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:24.095741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f142b-ea5d-430f-bf0a-61f1da74df4b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:24.096035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f142b-ea5d-430f-bf0a-61f1da74df4b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:24.096318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40f142b-ea5d-430f-bf0a-61f1da74df4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:24.097140 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f142b-ea5d-430f-bf0a-61f1da74df4b None admin] RBAC: Adding query filter params (interface=public, service_id=b5f68194ccf642fdbc3301dfea8d63fa, region_id=RegionOne) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:24.097496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f142b-ea5d-430f-bf0a-61f1da74df4b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:24.098961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40f142b-ea5d-430f-bf0a-61f1da74df4b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:24.109982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 465/908] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 5 15:01:23 2022] GET /identity/v3/endpoints?service_id=b5f68194ccf642fdbc3301dfea8d63fa&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:24.111382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a17a52-2f3b-4bd3-b8e3-28887c0cf2b1 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:24.210408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:24.217450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a17a52-2f3b-4bd3-b8e3-28887c0cf2b1 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:24.252674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d12ed0-756f-4dde-90db-2ad4d51aeef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:24.253268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d12ed0-756f-4dde-90db-2ad4d51aeef5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:24.253559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d12ed0-756f-4dde-90db-2ad4d51aeef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:24.253843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d12ed0-756f-4dde-90db-2ad4d51aeef5 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:24.254537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d12ed0-756f-4dde-90db-2ad4d51aeef5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:24.255941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d12ed0-756f-4dde-90db-2ad4d51aeef5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:24.267238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 444/909] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 5 15:01:24 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:01:24.866105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0b38d-9752-492a-acd1-a82a679b062c None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:24.866630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0b38d-9752-492a-acd1-a82a679b062c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:24.866630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0b38d-9752-492a-acd1-a82a679b062c None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:24.868306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 466/910] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:24.873416 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4d46-c1b9-4778-91cf-6f3c88f9ca22 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:24.873715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4d46-c1b9-4778-91cf-6f3c88f9ca22 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:24.873998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe4d46-c1b9-4778-91cf-6f3c88f9ca22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:24.891175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e6fe4d46-c1b9-4778-91cf-6f3c88f9ca22 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:24.892360 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e6fe4d46-c1b9-4778-91cf-6f3c88f9ca22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:24.959890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 445/911] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:24.982499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0b38d-9752-492a-acd1-a82a679b062c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:25.050055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d602cff-8e56-4b5a-ae09-d7224285fa24 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.050683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d602cff-8e56-4b5a-ae09-d7224285fa24 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.051244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d602cff-8e56-4b5a-ae09-d7224285fa24 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.053087 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 446/912] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:25.058144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0975305-0367-47b2-a85c-1d0acd2ba5a5 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.058659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0975305-0367-47b2-a85c-1d0acd2ba5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.059190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0975305-0367-47b2-a85c-1d0acd2ba5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.077505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d0975305-0367-47b2-a85c-1d0acd2ba5a5 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:25.079022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d0975305-0367-47b2-a85c-1d0acd2ba5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:25.097761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0b38d-9752-492a-acd1-a82a679b062c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:25.117039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:25.143109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d6bf9-0efb-4b04-988c-bd6e08276d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:25.143927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d6bf9-0efb-4b04-988c-bd6e08276d58 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.144209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d6bf9-0efb-4b04-988c-bd6e08276d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.144499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d6bf9-0efb-4b04-988c-bd6e08276d58 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.145755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d6bf9-0efb-4b04-988c-bd6e08276d58 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:25.147328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d6bf9-0efb-4b04-988c-bd6e08276d58 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:25.158249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 447/913] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-840d6bf9-0efb-4b04-988c-bd6e08276d58 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:25.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:25.161608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:25.162455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 467/914] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:01:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:25.167791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0975305-0367-47b2-a85c-1d0acd2ba5a5 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:25.181648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d6bf9-0efb-4b04-988c-bd6e08276d58 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:25.205890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:25.293218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0975305-0367-47b2-a85c-1d0acd2ba5a5 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:25.314204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d6bf9-0efb-4b04-988c-bd6e08276d58 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:25.337236 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220bf19-a1d9-4429-8425-7a0960900260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:25.338057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220bf19-a1d9-4429-8425-7a0960900260 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.338528 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220bf19-a1d9-4429-8425-7a0960900260 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.338914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3220bf19-a1d9-4429-8425-7a0960900260 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.340329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220bf19-a1d9-4429-8425-7a0960900260 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:25.340766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220bf19-a1d9-4429-8425-7a0960900260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:25.342510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3220bf19-a1d9-4429-8425-7a0960900260 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:25.349982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 448/915] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:01:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:25.355006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220bf19-a1d9-4429-8425-7a0960900260 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:25.359808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45134c-1ade-4d23-94a5-75d687221f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:25.360640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45134c-1ade-4d23-94a5-75d687221f74 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.360925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45134c-1ade-4d23-94a5-75d687221f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.361205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45134c-1ade-4d23-94a5-75d687221f74 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.372659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45134c-1ade-4d23-94a5-75d687221f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-8f45134c-1ade-4d23-94a5-75d687221f74 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:25.380835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:25.381772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:25.382620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:25.387479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 468/916] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 5 15:01:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:25.392177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45134c-1ade-4d23-94a5-75d687221f74 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:25.441470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:25.468291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:25.488134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3220bf19-a1d9-4429-8425-7a0960900260 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:25.502722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45134c-1ade-4d23-94a5-75d687221f74 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:25.527841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dc4f0-04ee-4e00-9d01-829936d554ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:25.529122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dc4f0-04ee-4e00-9d01-829936d554ff None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.529968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dc4f0-04ee-4e00-9d01-829936d554ff None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.530761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dc4f0-04ee-4e00-9d01-829936d554ff None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.531994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dc4f0-04ee-4e00-9d01-829936d554ff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:25.532902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dc4f0-04ee-4e00-9d01-829936d554ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:25.534840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dc4f0-04ee-4e00-9d01-829936d554ff None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:25.537349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa8c46-2bc1-465c-a9bf-251ba75d52d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:25.537870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa8c46-2bc1-465c-a9bf-251ba75d52d9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.538158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa8c46-2bc1-465c-a9bf-251ba75d52d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.538435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa8c46-2bc1-465c-a9bf-251ba75d52d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.539212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa8c46-2bc1-465c-a9bf-251ba75d52d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:25.539587 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa8c46-2bc1-465c-a9bf-251ba75d52d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:25.540872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa8c46-2bc1-465c-a9bf-251ba75d52d9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:25.542872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 449/917] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 5 15:01:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:25.548067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dc4f0-04ee-4e00-9d01-829936d554ff None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:25.554830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 469/918] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 5 15:01:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:25.555832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa8c46-2bc1-465c-a9bf-251ba75d52d9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:25.656301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:25.659815 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa8c46-2bc1-465c-a9bf-251ba75d52d9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:25.661103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dc4f0-04ee-4e00-9d01-829936d554ff None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:25.696425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088acbd-8e73-4731-8a38-38a7996c4638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:25.696623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f3af9-4ca0-4614-9b5a-b011fdc66e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:25.697074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088acbd-8e73-4731-8a38-38a7996c4638 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.697358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088acbd-8e73-4731-8a38-38a7996c4638 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.698014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f3af9-4ca0-4614-9b5a-b011fdc66e7e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.698187 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088acbd-8e73-4731-8a38-38a7996c4638 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.698284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f3af9-4ca0-4614-9b5a-b011fdc66e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.698481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f3af9-4ca0-4614-9b5a-b011fdc66e7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.698893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088acbd-8e73-4731-8a38-38a7996c4638 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:25.700389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088acbd-8e73-4731-8a38-38a7996c4638 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:25.709708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f3af9-4ca0-4614-9b5a-b011fdc66e7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:25.711216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f3af9-4ca0-4614-9b5a-b011fdc66e7e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:25.728189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 450/919] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 5 15:01:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:25.728189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6ba5-a81d-4aa7-9b25-dc32a80ff226 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.728189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6ba5-a81d-4aa7-9b25-dc32a80ff226 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.728189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6ba5-a81d-4aa7-9b25-dc32a80ff226 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.728189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 451/920] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:25.730995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec16430-5c18-4718-89d1-7c61ee25e788 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.731451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec16430-5c18-4718-89d1-7c61ee25e788 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.731803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec16430-5c18-4718-89d1-7c61ee25e788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.739259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-565f3af9-4ca0-4614-9b5a-b011fdc66e7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:25.743930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:25.744977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:25.745887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:25.745887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:25.745887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:25.745887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:25.748400 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 470/921] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:25.754198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f3af9-4ca0-4614-9b5a-b011fdc66e7e None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:25.756878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7ec16430-5c18-4718-89d1-7c61ee25e788 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:25.758356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7ec16430-5c18-4718-89d1-7c61ee25e788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:25.867186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 452/922] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:25.881868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f3af9-4ca0-4614-9b5a-b011fdc66e7e None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:25.892951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec16430-5c18-4718-89d1-7c61ee25e788 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:25.930312 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c293bb-1e63-4e32-8871-105d0b9450fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:25.930905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c293bb-1e63-4e32-8871-105d0b9450fa None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:25.931282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c293bb-1e63-4e32-8871-105d0b9450fa None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:25.931562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c293bb-1e63-4e32-8871-105d0b9450fa None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:25.932834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c293bb-1e63-4e32-8871-105d0b9450fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:25.933149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c293bb-1e63-4e32-8871-105d0b9450fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:25.934550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c293bb-1e63-4e32-8871-105d0b9450fa None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:25.949530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 471/923] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:25.955838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c293bb-1e63-4e32-8871-105d0b9450fa None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:25.972053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:26.028361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec16430-5c18-4718-89d1-7c61ee25e788 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:26.058494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:26.065889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879b136-625c-4149-9942-02c9b03fef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:26.066790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879b136-625c-4149-9942-02c9b03fef16 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:26.067313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879b136-625c-4149-9942-02c9b03fef16 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:26.067766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879b136-625c-4149-9942-02c9b03fef16 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:26.068673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879b136-625c-4149-9942-02c9b03fef16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:26.070215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a879b136-625c-4149-9942-02c9b03fef16 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:26.077547 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c293bb-1e63-4e32-8871-105d0b9450fa None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:26.086592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 453/924] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:01:25 2022] POST /identity/v3/services => generated 241 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:01:26.124233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d540d-fa97-4482-9e8e-8505ecb24f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:26.125072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d540d-fa97-4482-9e8e-8505ecb24f65 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:26.125357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d540d-fa97-4482-9e8e-8505ecb24f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:26.125634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d540d-fa97-4482-9e8e-8505ecb24f65 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:26.133980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d540d-fa97-4482-9e8e-8505ecb24f65 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:26.135598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d540d-fa97-4482-9e8e-8505ecb24f65 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-b45d540d-fa97-4482-9e8e-8505ecb24f65 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:26.153063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:26.154104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:26.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 472/925] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 5 15:01:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:26.162047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879b136-625c-4149-9942-02c9b03fef16 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:26.281161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879b136-625c-4149-9942-02c9b03fef16 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:26.318812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397e215-517c-4924-b9c1-9b365a4ef82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:26.319540 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397e215-517c-4924-b9c1-9b365a4ef82d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:26.319830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397e215-517c-4924-b9c1-9b365a4ef82d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:26.320108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397e215-517c-4924-b9c1-9b365a4ef82d None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:26.320957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397e215-517c-4924-b9c1-9b365a4ef82d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:26.321270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397e215-517c-4924-b9c1-9b365a4ef82d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:26.323248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397e215-517c-4924-b9c1-9b365a4ef82d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:26.342023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 454/926] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 5 15:01:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:26.343439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517442f-4ef2-40d1-bc76-cdb184241423 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:26.343731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517442f-4ef2-40d1-bc76-cdb184241423 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:26.344010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3517442f-4ef2-40d1-bc76-cdb184241423 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:26.345362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 473/927] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:26.349960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397e215-517c-4924-b9c1-9b365a4ef82d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:26.350304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f45b-d640-460c-8137-2931f839012c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:26.350687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f45b-d640-460c-8137-2931f839012c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:26.351078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6f45b-d640-460c-8137-2931f839012c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:26.372388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d8f6f45b-d640-460c-8137-2931f839012c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:26.373738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d8f6f45b-d640-460c-8137-2931f839012c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:26.421384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:26.442385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:26.450964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 474/928] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:26.472780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6f45b-d640-460c-8137-2931f839012c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:26.483839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397e215-517c-4924-b9c1-9b365a4ef82d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:26.529515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345edc4f-d805-432b-8313-662b7014cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:26.530567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345edc4f-d805-432b-8313-662b7014cb3e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:26.531171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345edc4f-d805-432b-8313-662b7014cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:26.531498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345edc4f-d805-432b-8313-662b7014cb3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:26.544273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345edc4f-d805-432b-8313-662b7014cb3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:26.545729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345edc4f-d805-432b-8313-662b7014cb3e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-345edc4f-d805-432b-8313-662b7014cb3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:26.563981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:26.564961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:26.565918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:26.565918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:26.565918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:26.565918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:26.565918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 455/929] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:26.570665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345edc4f-d805-432b-8313-662b7014cb3e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:26.593105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6f45b-d640-460c-8137-2931f839012c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:26.637473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60fee-4fe3-4668-a5f4-59fe77337196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:26.638459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60fee-4fe3-4668-a5f4-59fe77337196 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:26.638843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60fee-4fe3-4668-a5f4-59fe77337196 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:26.639257 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f60fee-4fe3-4668-a5f4-59fe77337196 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:26.640028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f60fee-4fe3-4668-a5f4-59fe77337196 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:26.641494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f60fee-4fe3-4668-a5f4-59fe77337196 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-f8f60fee-4fe3-4668-a5f4-59fe77337196 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:26.654344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:26.655540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 05 15:01:26.656575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:26.658714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 475/930] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Jul 5 15:01:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:26.664256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60fee-4fe3-4668-a5f4-59fe77337196 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:26.692981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345edc4f-d805-432b-8313-662b7014cb3e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:26.697663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:26.728399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:26.741693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b07790-5fda-4bc1-8669-e343dcf0a131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:26.742546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b07790-5fda-4bc1-8669-e343dcf0a131 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:26.742984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b07790-5fda-4bc1-8669-e343dcf0a131 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:26.744007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b07790-5fda-4bc1-8669-e343dcf0a131 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:26.745455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b07790-5fda-4bc1-8669-e343dcf0a131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:26.746429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b07790-5fda-4bc1-8669-e343dcf0a131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:26.748423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b07790-5fda-4bc1-8669-e343dcf0a131 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:26.768440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 456/931] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:26.770420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b07790-5fda-4bc1-8669-e343dcf0a131 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:26.777031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f60fee-4fe3-4668-a5f4-59fe77337196 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:26.812160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f1cc3-7c47-4647-8dae-e4d1028e5d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:26.812967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f1cc3-7c47-4647-8dae-e4d1028e5d9f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:26.813356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f1cc3-7c47-4647-8dae-e4d1028e5d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:26.813993 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f1cc3-7c47-4647-8dae-e4d1028e5d9f None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:26.814816 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f1cc3-7c47-4647-8dae-e4d1028e5d9f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:26.815302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f1cc3-7c47-4647-8dae-e4d1028e5d9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:26.816769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f1cc3-7c47-4647-8dae-e4d1028e5d9f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:26.827073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 476/932] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:26.832521 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f1cc3-7c47-4647-8dae-e4d1028e5d9f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:26.889708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b07790-5fda-4bc1-8669-e343dcf0a131 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:26.934669 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c9698-dba2-4a76-99fd-9b1c03a3c713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:26.936024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c9698-dba2-4a76-99fd-9b1c03a3c713 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:26.936744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c9698-dba2-4a76-99fd-9b1c03a3c713 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:26.937211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c9698-dba2-4a76-99fd-9b1c03a3c713 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:26.947279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c9698-dba2-4a76-99fd-9b1c03a3c713 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:26.950773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c9698-dba2-4a76-99fd-9b1c03a3c713 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:26.953436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f1cc3-7c47-4647-8dae-e4d1028e5d9f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c67c9698-dba2-4a76-99fd-9b1c03a3c713 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:26.970915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:26.972076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:26.973193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:26.973193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:26.973193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:26.973193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:26.973193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:26.973193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:26.973193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:26.973193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:26.981550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 457/933] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:26.983559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c9698-dba2-4a76-99fd-9b1c03a3c713 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:26.988897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:27.000041 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec265e7-fdf1-485b-a06c-04d1cf477ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:27.000769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec265e7-fdf1-485b-a06c-04d1cf477ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:27.001052 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec265e7-fdf1-485b-a06c-04d1cf477ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:27.001328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec265e7-fdf1-485b-a06c-04d1cf477ac3 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:27.002207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec265e7-fdf1-485b-a06c-04d1cf477ac3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:27.002400 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec265e7-fdf1-485b-a06c-04d1cf477ac3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:27.004047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec265e7-fdf1-485b-a06c-04d1cf477ac3 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:27.009259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 477/934] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:26 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:27.014332 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec265e7-fdf1-485b-a06c-04d1cf477ac3 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:27.016124 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:27.118761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c9698-dba2-4a76-99fd-9b1c03a3c713 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:27.130600 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec265e7-fdf1-485b-a06c-04d1cf477ac3 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:27.167096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6160-f505-4ef3-9bef-ad7123f38059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:27.168308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6160-f505-4ef3-9bef-ad7123f38059 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:27.168744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6160-f505-4ef3-9bef-ad7123f38059 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:27.169196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc6160-f505-4ef3-9bef-ad7123f38059 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:27.170138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6160-f505-4ef3-9bef-ad7123f38059 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:27.170599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6160-f505-4ef3-9bef-ad7123f38059 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:27.170958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d8ea6-a17d-4bb7-b95d-22df2da69bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:27.171736 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d8ea6-a17d-4bb7-b95d-22df2da69bfc None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:27.172115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d8ea6-a17d-4bb7-b95d-22df2da69bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:27.172501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d8ea6-a17d-4bb7-b95d-22df2da69bfc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:27.172876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc6160-f505-4ef3-9bef-ad7123f38059 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:27.173431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d8ea6-a17d-4bb7-b95d-22df2da69bfc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:27.175213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902d8ea6-a17d-4bb7-b95d-22df2da69bfc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:27.189242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 458/935] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:27.201882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 478/936] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 5 15:01:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:27.205225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6160-f505-4ef3-9bef-ad7123f38059 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:27.311533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:27.332823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc6160-f505-4ef3-9bef-ad7123f38059 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:27.374755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dfdc9-349e-4bc4-9c38-27bd5e59fde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:27.375669 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dfdc9-349e-4bc4-9c38-27bd5e59fde1 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:27.375950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dfdc9-349e-4bc4-9c38-27bd5e59fde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:27.376223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dfdc9-349e-4bc4-9c38-27bd5e59fde1 None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/b3ab5ed3d9e341f7853302ac7826879b/roles/e08bb5ad422e4f6593bcbaca2bdf4366` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:27.378402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dfdc9-349e-4bc4-9c38-27bd5e59fde1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=b3ab5ed3d9e341f7853302ac7826879b, role_id=e08bb5ad422e4f6593bcbaca2bdf4366)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:27.379859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dfdc9-349e-4bc4-9c38-27bd5e59fde1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:27.394710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 459/937] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:01:27 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/b3ab5ed3d9e341f7853302ac7826879b/roles/e08bb5ad422e4f6593bcbaca2bdf4366 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:01:27.935350 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a00fa-65f2-4637-a04c-137dd854208b None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:27.935734 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a00fa-65f2-4637-a04c-137dd854208b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:27.935946 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a00fa-65f2-4637-a04c-137dd854208b None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:27.937504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 479/938] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:27.942712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d914b46-68c4-4fec-a7ab-395c0c6d10cb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:27.943011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d914b46-68c4-4fec-a7ab-395c0c6d10cb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:27.943325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d914b46-68c4-4fec-a7ab-395c0c6d10cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:27.961059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4d914b46-68c4-4fec-a7ab-395c0c6d10cb None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:27.962250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4d914b46-68c4-4fec-a7ab-395c0c6d10cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:28.034328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 460/939] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:28.057713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a00fa-65f2-4637-a04c-137dd854208b None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:28.108753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:28.168812 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a00fa-65f2-4637-a04c-137dd854208b None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:28.172888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0413-50cb-4610-852f-5eac214615fe None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:28.173330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0413-50cb-4610-852f-5eac214615fe None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:28.173511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0413-50cb-4610-852f-5eac214615fe None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:28.175091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 461/940] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:28.179875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e762b-46e1-46d3-8d27-2ef7a4b1f465 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:28.180230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e762b-46e1-46d3-8d27-2ef7a4b1f465 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:28.180582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e762b-46e1-46d3-8d27-2ef7a4b1f465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:28.197816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ce5e762b-46e1-46d3-8d27-2ef7a4b1f465 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:28.199349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ce5e762b-46e1-46d3-8d27-2ef7a4b1f465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:28.207396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:28.208309 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:28.208745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:28.209163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:28.210115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:28.211849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:28.213912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: WARNING keystone.server.flask.common [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 05 15:01:28.234316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.sql.core [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 05 15:01:28.234316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 05 15:01:28.234316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [parameters: {'id': 'd9e9240a8af949c7a5ce7c2eb25ddb5e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 05 15:01:28.234316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80351) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9e9240a8af949c7a5ce7c2eb25ddb5e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9e9240a8af949c7a5ce7c2eb25ddb5e. Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:01:28.252934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application result.read() Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 05 15:01:28.253798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application yield resource Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self.session.rollback() Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self.force_reraise() Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise self.value Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application session.commit() Jul 05 15:01:28.254774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._prepare_impl() Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self.session.flush() Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._flush(objects) Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application compat.raise_( Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application flush_context.execute() Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 05 15:01:28.255709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rec.execute(self) Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 05 15:01:28.256610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application result.read() Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:01:28.257416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [parameters: {'id': 'd9e9240a8af949c7a5ce7c2eb25ddb5e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:28.258299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:28.259285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:28.260315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:28.260315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 05 15:01:28.260315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 05 15:01:28.260315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:28.260315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:28.260315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 05 15:01:28.260315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 05 15:01:28.260315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9e9240a8af949c7a5ce7c2eb25ddb5e. Jul 05 15:01:28.260315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:28.262541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 480/941] 10.222.0.23 () {60 vars in 1140 bytes} [Tue Jul 5 15:01:28 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:28.271170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:28.271850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:28.284445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 462/942] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:28.307329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e762b-46e1-46d3-8d27-2ef7a4b1f465 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:28.381834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643739-a7b7-4190-849c-d03910d2ec67 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:28.383584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:28.418801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bc958-8479-4270-81b8-9ada2ac2bf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:28.419691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bc958-8479-4270-81b8-9ada2ac2bf14 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:28.419965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bc958-8479-4270-81b8-9ada2ac2bf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:28.420292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bc958-8479-4270-81b8-9ada2ac2bf14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:28.422672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e762b-46e1-46d3-8d27-2ef7a4b1f465 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:28.428279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bc958-8479-4270-81b8-9ada2ac2bf14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:28.430081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bc958-8479-4270-81b8-9ada2ac2bf14 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-326bc958-8479-4270-81b8-9ada2ac2bf14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:28.445713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:28.446860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:28.448622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:28.448622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:28.448622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:28.448622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:28.448622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:28.448622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:28.448622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:28.448622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:28.451288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 481/943] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:28.453590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bc958-8479-4270-81b8-9ada2ac2bf14 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:28.469068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83058657-d053-479c-af7c-24f9882d2ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:28.470328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83058657-d053-479c-af7c-24f9882d2ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:28.470770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83058657-d053-479c-af7c-24f9882d2ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:28.471350 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83058657-d053-479c-af7c-24f9882d2ef4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:28.472241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83058657-d053-479c-af7c-24f9882d2ef4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:28.473878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83058657-d053-479c-af7c-24f9882d2ef4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-83058657-d053-479c-af7c-24f9882d2ef4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:28.487954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:28.489266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:28.490408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:28.493059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 463/944] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:28.503067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83058657-d053-479c-af7c-24f9882d2ef4 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:28.568579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bc958-8479-4270-81b8-9ada2ac2bf14 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:28.575979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:28.606829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02029d6f-a767-458e-81c6-2caeb0b495fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:28.607871 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02029d6f-a767-458e-81c6-2caeb0b495fa None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:28.608299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02029d6f-a767-458e-81c6-2caeb0b495fa None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:28.608790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02029d6f-a767-458e-81c6-2caeb0b495fa None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:28.609711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02029d6f-a767-458e-81c6-2caeb0b495fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:28.610172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02029d6f-a767-458e-81c6-2caeb0b495fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:28.611890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02029d6f-a767-458e-81c6-2caeb0b495fa None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:28.624445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83058657-d053-479c-af7c-24f9882d2ef4 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:28.645318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 482/945] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:28.660718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4c451-0cf9-40ec-9f38-5994d3d2f3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:28.661402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4c451-0cf9-40ec-9f38-5994d3d2f3bb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:28.661685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4c451-0cf9-40ec-9f38-5994d3d2f3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:28.661959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4c451-0cf9-40ec-9f38-5994d3d2f3bb None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:28.662683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4c451-0cf9-40ec-9f38-5994d3d2f3bb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:28.662990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4c451-0cf9-40ec-9f38-5994d3d2f3bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:28.664415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4c451-0cf9-40ec-9f38-5994d3d2f3bb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:28.671219 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 464/946] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 5 15:01:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:28.676744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02029d6f-a767-458e-81c6-2caeb0b495fa None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:28.719631 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:28.792872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02029d6f-a767-458e-81c6-2caeb0b495fa None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:28.827000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2ae16-e687-40df-9a9b-20b11f720bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:28.827694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2ae16-e687-40df-9a9b-20b11f720bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:28.827979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2ae16-e687-40df-9a9b-20b11f720bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:28.828252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a2ae16-e687-40df-9a9b-20b11f720bd8 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:28.828964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2ae16-e687-40df-9a9b-20b11f720bd8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:28.829272 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2ae16-e687-40df-9a9b-20b11f720bd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:28.830664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a2ae16-e687-40df-9a9b-20b11f720bd8 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:28.836187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 483/947] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 5 15:01:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:28.842891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4c451-0cf9-40ec-9f38-5994d3d2f3bb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:28.949502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4c451-0cf9-40ec-9f38-5994d3d2f3bb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:28.974498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:28.986410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb2998-0e7a-4020-b8d2-2a00db9a6ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:28.987165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb2998-0e7a-4020-b8d2-2a00db9a6ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:28.987450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb2998-0e7a-4020-b8d2-2a00db9a6ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:28.987732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb2998-0e7a-4020-b8d2-2a00db9a6ec5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:28.988531 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb2998-0e7a-4020-b8d2-2a00db9a6ec5 None admin] RBAC: Adding query filter params (interface=public, service_id=b5edce12426d40dea1a4d7fb847407c3, region_id=RegionOne) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:28.988841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb2998-0e7a-4020-b8d2-2a00db9a6ec5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:28.990206 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb2998-0e7a-4020-b8d2-2a00db9a6ec5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:28.996579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 465/948] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 5 15:01:28 2022] GET /identity/v3/endpoints?service_id=b5edce12426d40dea1a4d7fb847407c3&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:29.001952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2ae16-e687-40df-9a9b-20b11f720bd8 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:29.094383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:29.111172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a2ae16-e687-40df-9a9b-20b11f720bd8 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:29.146681 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95487daf-b342-435c-ac24-2d670eb1d2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:29.147762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95487daf-b342-435c-ac24-2d670eb1d2aa None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:29.148271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95487daf-b342-435c-ac24-2d670eb1d2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:29.148776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95487daf-b342-435c-ac24-2d670eb1d2aa None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:29.149665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95487daf-b342-435c-ac24-2d670eb1d2aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:29.151297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95487daf-b342-435c-ac24-2d670eb1d2aa None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:29.157862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 484/949] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 5 15:01:29 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:01:29.393543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830c052-a06d-4951-bfbb-47e72d567757 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:29.394080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830c052-a06d-4951-bfbb-47e72d567757 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:29.394080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830c052-a06d-4951-bfbb-47e72d567757 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:29.395767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 466/950] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:29.401095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82069b9-4188-446f-a970-7acdd82ac8f0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:29.401390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82069b9-4188-446f-a970-7acdd82ac8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:29.401674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82069b9-4188-446f-a970-7acdd82ac8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:29.419646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b82069b9-4188-446f-a970-7acdd82ac8f0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:29.421166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b82069b9-4188-446f-a970-7acdd82ac8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:29.486258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735daf1-943e-4c1e-8066-b8dfa5543f7f None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:29.486602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735daf1-943e-4c1e-8066-b8dfa5543f7f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:29.486828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735daf1-943e-4c1e-8066-b8dfa5543f7f None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:29.488772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 467/951] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:29.489947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 485/952] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:29.493669 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c01c29-7386-4a98-a3a5-0ee6dbba95f7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:29.493958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c01c29-7386-4a98-a3a5-0ee6dbba95f7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:29.494814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c01c29-7386-4a98-a3a5-0ee6dbba95f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:29.512135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-79c01c29-7386-4a98-a3a5-0ee6dbba95f7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:29.512694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82069b9-4188-446f-a970-7acdd82ac8f0 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:29.513475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-79c01c29-7386-4a98-a3a5-0ee6dbba95f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:29.594081 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 468/953] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:29.610050 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:29.617406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c01c29-7386-4a98-a3a5-0ee6dbba95f7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:29.633461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82069b9-4188-446f-a970-7acdd82ac8f0 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:29.679403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2993862-14cf-40ea-829c-0ef89c8b43cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:29.680686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2993862-14cf-40ea-829c-0ef89c8b43cc None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:29.682171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2993862-14cf-40ea-829c-0ef89c8b43cc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:29.682682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2993862-14cf-40ea-829c-0ef89c8b43cc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:29.683627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2993862-14cf-40ea-829c-0ef89c8b43cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:29.686439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2993862-14cf-40ea-829c-0ef89c8b43cc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-e2993862-14cf-40ea-829c-0ef89c8b43cc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:29.705122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:29.706172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:01:29.707237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:29.708687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 486/954] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:29.718957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2993862-14cf-40ea-829c-0ef89c8b43cc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:29.732443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c01c29-7386-4a98-a3a5-0ee6dbba95f7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:29.751740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:29.772137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88505599-097f-4915-9aa6-805cf93560f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:29.772961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88505599-097f-4915-9aa6-805cf93560f9 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:29.773244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88505599-097f-4915-9aa6-805cf93560f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:29.773516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88505599-097f-4915-9aa6-805cf93560f9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:29.784036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88505599-097f-4915-9aa6-805cf93560f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-88505599-097f-4915-9aa6-805cf93560f9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:29.792371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:29.793377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:29.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:29.798588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 469/955] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 5 15:01:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:29.799576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88505599-097f-4915-9aa6-805cf93560f9 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:29.851518 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2993862-14cf-40ea-829c-0ef89c8b43cc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:29.894789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b47f5-9d76-4b3a-a43a-7a5aa33c76df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:29.895921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b47f5-9d76-4b3a-a43a-7a5aa33c76df None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:29.897224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b47f5-9d76-4b3a-a43a-7a5aa33c76df None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:29.897771 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b47f5-9d76-4b3a-a43a-7a5aa33c76df None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:29.899330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b47f5-9d76-4b3a-a43a-7a5aa33c76df None admin] RBAC: Adding query filter params (name=compute) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:29.899901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b47f5-9d76-4b3a-a43a-7a5aa33c76df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:29.902884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b47f5-9d76-4b3a-a43a-7a5aa33c76df None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:29.915773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 487/956] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:29.917449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b47f5-9d76-4b3a-a43a-7a5aa33c76df None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:29.930432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88505599-097f-4915-9aa6-805cf93560f9 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:29.935247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:29.976174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a31a1-b5bd-45a3-958a-306d2a7cd5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:29.977654 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a31a1-b5bd-45a3-958a-306d2a7cd5bf None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:29.978166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a31a1-b5bd-45a3-958a-306d2a7cd5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:29.979203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a31a1-b5bd-45a3-958a-306d2a7cd5bf None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:29.980947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a31a1-b5bd-45a3-958a-306d2a7cd5bf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:29.981465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a31a1-b5bd-45a3-958a-306d2a7cd5bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:29.983953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a31a1-b5bd-45a3-958a-306d2a7cd5bf None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:29.996444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 470/957] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 5 15:01:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:30.002983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a31a1-b5bd-45a3-958a-306d2a7cd5bf None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:30.051648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:30.053348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b47f5-9d76-4b3a-a43a-7a5aa33c76df None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:30.088829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c806fc-333f-4779-b666-57329f23771c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:30.090034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c806fc-333f-4779-b666-57329f23771c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:30.090550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c806fc-333f-4779-b666-57329f23771c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:30.091107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c806fc-333f-4779-b666-57329f23771c None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:30.092101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c806fc-333f-4779-b666-57329f23771c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:30.092612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c806fc-333f-4779-b666-57329f23771c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:30.094194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c806fc-333f-4779-b666-57329f23771c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:30.099401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 488/958] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:30.131139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a31a1-b5bd-45a3-958a-306d2a7cd5bf None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:30.169509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88302af9-9252-4ad3-95e1-1dd29c0e4752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:30.170731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302af9-9252-4ad3-95e1-1dd29c0e4752 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:30.171261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302af9-9252-4ad3-95e1-1dd29c0e4752 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:30.171775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88302af9-9252-4ad3-95e1-1dd29c0e4752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:30.184387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88302af9-9252-4ad3-95e1-1dd29c0e4752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:30.186185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88302af9-9252-4ad3-95e1-1dd29c0e4752 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-88302af9-9252-4ad3-95e1-1dd29c0e4752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:30.209019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:30.210089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:30.211001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:30.211001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:30.211001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:30.211001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:30.211001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 471/959] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:30.216881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c806fc-333f-4779-b666-57329f23771c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:30.306661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:30.342633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c806fc-333f-4779-b666-57329f23771c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:30.377679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48250644-703a-4dfb-a71c-51a7bd07871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:30.378356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250644-703a-4dfb-a71c-51a7bd07871f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:30.378644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250644-703a-4dfb-a71c-51a7bd07871f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:30.378923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48250644-703a-4dfb-a71c-51a7bd07871f None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:30.379695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48250644-703a-4dfb-a71c-51a7bd07871f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:30.380004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48250644-703a-4dfb-a71c-51a7bd07871f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:30.381386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48250644-703a-4dfb-a71c-51a7bd07871f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:30.395014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 489/960] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:30.400578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88302af9-9252-4ad3-95e1-1dd29c0e4752 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:30.498012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:30.507243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88302af9-9252-4ad3-95e1-1dd29c0e4752 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:30.543636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077eba62-c6d9-46fd-9238-fc7721f64618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:30.544451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077eba62-c6d9-46fd-9238-fc7721f64618 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:30.544743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077eba62-c6d9-46fd-9238-fc7721f64618 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:30.545020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077eba62-c6d9-46fd-9238-fc7721f64618 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:30.552336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077eba62-c6d9-46fd-9238-fc7721f64618 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:30.553780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077eba62-c6d9-46fd-9238-fc7721f64618 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-077eba62-c6d9-46fd-9238-fc7721f64618 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:30.569082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:30.570075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:30.571032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 472/961] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 5 15:01:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:30.576703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48250644-703a-4dfb-a71c-51a7bd07871f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:30.683825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48250644-703a-4dfb-a71c-51a7bd07871f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:30.715417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:30.720937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55062283-ede2-489b-a979-1398cbe5e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:30.722017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55062283-ede2-489b-a979-1398cbe5e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:30.722572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55062283-ede2-489b-a979-1398cbe5e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:30.723220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55062283-ede2-489b-a979-1398cbe5e5a1 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:30.724074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55062283-ede2-489b-a979-1398cbe5e5a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:30.724388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55062283-ede2-489b-a979-1398cbe5e5a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:30.725831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55062283-ede2-489b-a979-1398cbe5e5a1 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:30.744017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 490/962] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 5 15:01:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:30.749627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077eba62-c6d9-46fd-9238-fc7721f64618 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:30.825191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc67d8-8e17-4262-bf43-33230e878e0b None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:30.825767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc67d8-8e17-4262-bf43-33230e878e0b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:30.826001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc67d8-8e17-4262-bf43-33230e878e0b None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:30.827662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 491/963] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:30.832401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eef5cd-04b3-4e05-8bd9-436e30ec5c51 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:30.832693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eef5cd-04b3-4e05-8bd9-436e30ec5c51 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:30.832974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eef5cd-04b3-4e05-8bd9-436e30ec5c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:30.849690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b6eef5cd-04b3-4e05-8bd9-436e30ec5c51 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:30.850892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b6eef5cd-04b3-4e05-8bd9-436e30ec5c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:30.861205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077eba62-c6d9-46fd-9238-fc7721f64618 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:30.900889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c0865-b126-43db-9a0a-b8eeeadd2b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:30.901542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c0865-b126-43db-9a0a-b8eeeadd2b85 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:30.901825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c0865-b126-43db-9a0a-b8eeeadd2b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:30.902107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8c0865-b126-43db-9a0a-b8eeeadd2b85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:30.921901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 492/964] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:30.931965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:30.950110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eef5cd-04b3-4e05-8bd9-436e30ec5c51 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:30.985738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c0865-b126-43db-9a0a-b8eeeadd2b85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:30.987376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8c0865-b126-43db-9a0a-b8eeeadd2b85 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:31.049544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eef5cd-04b3-4e05-8bd9-436e30ec5c51 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:31.070851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:31.085786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf6984-06cb-4893-80e2-e1c06855df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:31.086475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf6984-06cb-4893-80e2-e1c06855df14 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.086769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf6984-06cb-4893-80e2-e1c06855df14 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.087066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf6984-06cb-4893-80e2-e1c06855df14 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.087852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf6984-06cb-4893-80e2-e1c06855df14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:31.089614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcf6984-06cb-4893-80e2-e1c06855df14 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:31.104130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.sql.core [None req-8dcf6984-06cb-4893-80e2-e1c06855df14 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 05 15:01:31.104130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 05 15:01:31.104130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [parameters: {'user_id': '7bce1b906a3b4180a5851b2023e44829', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 05 15:01:31.104130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80351) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 05 15:01:31.107591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 473/965] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:30 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:01:31.112975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c0865-b126-43db-9a0a-b8eeeadd2b85 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-8dcf6984-06cb-4893-80e2-e1c06855df14 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application result.read() Jul 05 15:01:31.116686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:01:31.117727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application yield resource Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application next(self.gen) Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self.session.rollback() Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self.force_reraise() Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise self.value Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application session.commit() Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 05 15:01:31.118738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._prepare_impl() Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self.session.flush() Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._flush(objects) Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application compat.raise_( Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application flush_context.execute() Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rec.execute(self) Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 05 15:01:31.119890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 05 15:01:31.121027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application result = self._query(query) Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application conn.query(q) Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application result.read() Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 05 15:01:31.122101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [parameters: {'user_id': '7bce1b906a3b4180a5851b2023e44829', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:31.123186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 05 15:01:31.127389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 05 15:01:31.128376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 05 15:01:31.128376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 05 15:01:31.128376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 05 15:01:31.128376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 05 15:01:31.128376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 05 15:01:31.128376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:31.128376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 493/966] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jul 5 15:01:30 2022] POST /identity/v3/users => generated 160 bytes in 169 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:31.128376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf6984-06cb-4893-80e2-e1c06855df14 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:31.180945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:31.241050 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcf6984-06cb-4893-80e2-e1c06855df14 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:31.243950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8c0865-b126-43db-9a0a-b8eeeadd2b85 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:31.276610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb21810-b43e-4ddd-a522-1ad969090a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:31.277465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb21810-b43e-4ddd-a522-1ad969090a62 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.277690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb21810-b43e-4ddd-a522-1ad969090a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.277966 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb21810-b43e-4ddd-a522-1ad969090a62 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.280707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9d94a0-6f4a-4f79-9b08-20bc4006edc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:31.281492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d94a0-6f4a-4f79-9b08-20bc4006edc6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.281774 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d94a0-6f4a-4f79-9b08-20bc4006edc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.282049 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9d94a0-6f4a-4f79-9b08-20bc4006edc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.286419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb21810-b43e-4ddd-a522-1ad969090a62 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:31.288084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb21810-b43e-4ddd-a522-1ad969090a62 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:31.293550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d94a0-6f4a-4f79-9b08-20bc4006edc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:31.295009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9d94a0-6f4a-4f79-9b08-20bc4006edc6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-beb21810-b43e-4ddd-a522-1ad969090a62 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:31.302172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:31.303163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:31.304156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 494/967] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:01:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:31.304786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304780f0-a492-4f60-8e19-6e9c8302c1ee None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.304867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304780f0-a492-4f60-8e19-6e9c8302c1ee None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.305832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304780f0-a492-4f60-8e19-6e9c8302c1ee None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.309794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 495/968] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:31.310515 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304780f0-a492-4f60-8e19-6e9c8302c1ee None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-2f9d94a0-6f4a-4f79-9b08-20bc4006edc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:31.318171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:31.319489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:31.320573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:31.320573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:31.320573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:31.320573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:31.323465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 474/969] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:31.327405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2648b-e832-4664-aa26-0d642d400107 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.327405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2648b-e832-4664-aa26-0d642d400107 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.327405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2648b-e832-4664-aa26-0d642d400107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.343968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-24c2648b-e832-4664-aa26-0d642d400107 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:31.345257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-24c2648b-e832-4664-aa26-0d642d400107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:31.359323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:31.431065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 475/970] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:31.431065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2648b-e832-4664-aa26-0d642d400107 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:31.431713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304780f0-a492-4f60-8e19-6e9c8302c1ee None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:31.460030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:31.464818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e920a-4be8-46c8-9ad3-2feecd47615e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:31.465586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e920a-4be8-46c8-9ad3-2feecd47615e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.465877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e920a-4be8-46c8-9ad3-2feecd47615e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.466158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105e920a-4be8-46c8-9ad3-2feecd47615e None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.467003 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e920a-4be8-46c8-9ad3-2feecd47615e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:31.467348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e920a-4be8-46c8-9ad3-2feecd47615e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:31.468836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105e920a-4be8-46c8-9ad3-2feecd47615e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:31.503881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 496/971] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 5 15:01:31 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:31.503881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e920a-4be8-46c8-9ad3-2feecd47615e None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:31.542912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2648b-e832-4664-aa26-0d642d400107 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:31.581779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66bf7b-eb9c-4404-8218-79c45f06eeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:31.582636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66bf7b-eb9c-4404-8218-79c45f06eeca None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.583162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66bf7b-eb9c-4404-8218-79c45f06eeca None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.583512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d66bf7b-eb9c-4404-8218-79c45f06eeca None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.584342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66bf7b-eb9c-4404-8218-79c45f06eeca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:31.584726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66bf7b-eb9c-4404-8218-79c45f06eeca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:31.586309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d66bf7b-eb9c-4404-8218-79c45f06eeca None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:31.600506 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 476/972] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:31.608276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66bf7b-eb9c-4404-8218-79c45f06eeca None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:31.630041 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:31.633459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105e920a-4be8-46c8-9ad3-2feecd47615e None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:31.679781 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b00701-2f63-4077-aaf4-f62cbe3eb38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:31.680954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b00701-2f63-4077-aaf4-f62cbe3eb38c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.681439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b00701-2f63-4077-aaf4-f62cbe3eb38c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.682540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b00701-2f63-4077-aaf4-f62cbe3eb38c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.683490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b00701-2f63-4077-aaf4-f62cbe3eb38c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:31.684934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b00701-2f63-4077-aaf4-f62cbe3eb38c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-35b00701-2f63-4077-aaf4-f62cbe3eb38c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:31.699190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:31.700728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:31.701898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 497/973] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:31.706704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b00701-2f63-4077-aaf4-f62cbe3eb38c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:31.722611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:31.731447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d66bf7b-eb9c-4404-8218-79c45f06eeca None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:31.776550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8052c12-23cd-4a5c-a1a4-b779f882471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:31.777354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8052c12-23cd-4a5c-a1a4-b779f882471f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.777630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8052c12-23cd-4a5c-a1a4-b779f882471f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.779140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8052c12-23cd-4a5c-a1a4-b779f882471f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.790172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8052c12-23cd-4a5c-a1a4-b779f882471f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:31.792406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8052c12-23cd-4a5c-a1a4-b779f882471f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c8052c12-23cd-4a5c-a1a4-b779f882471f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:31.808851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:31.809822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:31.810753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:31.810753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:31.810753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:31.810753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:31.810753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:31.810753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:31.810753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:31.810753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:31.810753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 477/974] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:31.815744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8052c12-23cd-4a5c-a1a4-b779f882471f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:31.837958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b00701-2f63-4077-aaf4-f62cbe3eb38c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:31.876642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e16c28-54da-46b6-98ac-751b2b123c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:31.877378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e16c28-54da-46b6-98ac-751b2b123c75 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.877639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e16c28-54da-46b6-98ac-751b2b123c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.877920 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e16c28-54da-46b6-98ac-751b2b123c75 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.878638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e16c28-54da-46b6-98ac-751b2b123c75 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:31.878951 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e16c28-54da-46b6-98ac-751b2b123c75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:31.880378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e16c28-54da-46b6-98ac-751b2b123c75 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:31.886144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 498/975] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 5 15:01:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:31.891530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e16c28-54da-46b6-98ac-751b2b123c75 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:31.921133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:31.926966 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8052c12-23cd-4a5c-a1a4-b779f882471f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:31.964727 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b282b8-d93c-438f-bdda-16df0f1edf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:31.965389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b282b8-d93c-438f-bdda-16df0f1edf42 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:31.965784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b282b8-d93c-438f-bdda-16df0f1edf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:31.966066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b282b8-d93c-438f-bdda-16df0f1edf42 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:31.966855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b282b8-d93c-438f-bdda-16df0f1edf42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:31.967210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b282b8-d93c-438f-bdda-16df0f1edf42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:31.968647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b282b8-d93c-438f-bdda-16df0f1edf42 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:31.986135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 478/976] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:31.987160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b282b8-d93c-438f-bdda-16df0f1edf42 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:32.002284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:32.006300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e16c28-54da-46b6-98ac-751b2b123c75 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:32.041879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1bac1-f9af-490a-8697-a6c6ff3c868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:32.042557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1bac1-f9af-490a-8697-a6c6ff3c868a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:32.042848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1bac1-f9af-490a-8697-a6c6ff3c868a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:32.043230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1bac1-f9af-490a-8697-a6c6ff3c868a None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:32.043939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1bac1-f9af-490a-8697-a6c6ff3c868a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:32.044253 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1bac1-f9af-490a-8697-a6c6ff3c868a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:32.045632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1bac1-f9af-490a-8697-a6c6ff3c868a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:32.050477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 499/977] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 5 15:01:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:32.055952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1bac1-f9af-490a-8697-a6c6ff3c868a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:32.096097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b282b8-d93c-438f-bdda-16df0f1edf42 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:32.130993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5802f39c-c055-4657-896b-2f0e3ab4b733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:32.131696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802f39c-c055-4657-896b-2f0e3ab4b733 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:32.131978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802f39c-c055-4657-896b-2f0e3ab4b733 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:32.132261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5802f39c-c055-4657-896b-2f0e3ab4b733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:32.133111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802f39c-c055-4657-896b-2f0e3ab4b733 None admin] RBAC: Adding query filter params (role.id=e08bb5ad422e4f6593bcbaca2bdf4366, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=b3ab5ed3d9e341f7853302ac7826879b) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:32.133430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802f39c-c055-4657-896b-2f0e3ab4b733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:32.134987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5802f39c-c055-4657-896b-2f0e3ab4b733 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:32.141104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 479/978] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:31 2022] GET /identity/v3/role_assignments?user.id=b3ab5ed3d9e341f7853302ac7826879b&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=e08bb5ad422e4f6593bcbaca2bdf4366 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:32.161732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:32.173118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1bac1-f9af-490a-8697-a6c6ff3c868a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:32.210927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19babc5-f7d4-42ed-8c65-a97bda82d61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:32.211640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19babc5-f7d4-42ed-8c65-a97bda82d61e None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:32.211929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19babc5-f7d4-42ed-8c65-a97bda82d61e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:32.212202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19babc5-f7d4-42ed-8c65-a97bda82d61e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:32.213506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19babc5-f7d4-42ed-8c65-a97bda82d61e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:32.214918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19babc5-f7d4-42ed-8c65-a97bda82d61e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:32.236877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 500/979] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 5 15:01:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:01:32.272666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e097a07-8537-402c-a945-c546219aabd7 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:32.273059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e097a07-8537-402c-a945-c546219aabd7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:32.273414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e097a07-8537-402c-a945-c546219aabd7 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:32.276004 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 480/980] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:32.281866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9e201-e14e-4fc8-97ee-a5df529235b0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:32.282216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9e201-e14e-4fc8-97ee-a5df529235b0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:32.282503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f9e201-e14e-4fc8-97ee-a5df529235b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:32.305355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-70f9e201-e14e-4fc8-97ee-a5df529235b0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:32.308041 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-70f9e201-e14e-4fc8-97ee-a5df529235b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:32.391897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 501/981] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:32.414110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e097a07-8537-402c-a945-c546219aabd7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:32.496397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:32.520246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e097a07-8537-402c-a945-c546219aabd7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:32.554058 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504b5df-bd21-44b5-ba4d-b1046d873448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:32.554826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504b5df-bd21-44b5-ba4d-b1046d873448 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:32.555208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504b5df-bd21-44b5-ba4d-b1046d873448 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:32.555563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504b5df-bd21-44b5-ba4d-b1046d873448 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:32.556370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504b5df-bd21-44b5-ba4d-b1046d873448 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:32.558368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504b5df-bd21-44b5-ba4d-b1046d873448 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:32.571925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 481/982] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:01:32 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:34.197886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cc31e-e2b2-4990-9aee-ee9e87bab287 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.198315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cc31e-e2b2-4990-9aee-ee9e87bab287 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.198659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cc31e-e2b2-4990-9aee-ee9e87bab287 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.202070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 502/983] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:34.206550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbe2f5-49bb-4452-a7e8-9fbd7601b0e0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.206983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbe2f5-49bb-4452-a7e8-9fbd7601b0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.207189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbe2f5-49bb-4452-a7e8-9fbd7601b0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.225026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-17fbe2f5-49bb-4452-a7e8-9fbd7601b0e0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:34.226215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-17fbe2f5-49bb-4452-a7e8-9fbd7601b0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:34.279434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1b7a2-a6d9-48df-8f8d-ae91de0dac9a None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.279755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1b7a2-a6d9-48df-8f8d-ae91de0dac9a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.280018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1b7a2-a6d9-48df-8f8d-ae91de0dac9a None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.281655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 503/984] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:34.286505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8ba9-75a5-4fb8-8c3f-bd7d34ec8749 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.286800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8ba9-75a5-4fb8-8c3f-bd7d34ec8749 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.287101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c8ba9-75a5-4fb8-8c3f-bd7d34ec8749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.303327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 482/985] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:34.304537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-e90c8ba9-75a5-4fb8-8c3f-bd7d34ec8749 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:34.305737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e90c8ba9-75a5-4fb8-8c3f-bd7d34ec8749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:34.325359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbe2f5-49bb-4452-a7e8-9fbd7601b0e0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:34.382355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 504/986] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:34.388801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:34.404893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c8ba9-75a5-4fb8-8c3f-bd7d34ec8749 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:34.407936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:34.430894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbe2f5-49bb-4452-a7e8-9fbd7601b0e0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:34.469771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03ca9c-61ef-47f8-aef1-693ada88c7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:34.470482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03ca9c-61ef-47f8-aef1-693ada88c7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.470883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03ca9c-61ef-47f8-aef1-693ada88c7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.471189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03ca9c-61ef-47f8-aef1-693ada88c7e5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.471846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03ca9c-61ef-47f8-aef1-693ada88c7e5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:34.473232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03ca9c-61ef-47f8-aef1-693ada88c7e5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4a03ca9c-61ef-47f8-aef1-693ada88c7e5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:34.485608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:34.486645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:34.487550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 483/987] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Jul 5 15:01:34 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:34.492732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03ca9c-61ef-47f8-aef1-693ada88c7e5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:34.512559 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c8ba9-75a5-4fb8-8c3f-bd7d34ec8749 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:34.546974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a6d20-34e4-4533-980a-07c2f343e6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:34.547813 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a6d20-34e4-4533-980a-07c2f343e6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.548099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a6d20-34e4-4533-980a-07c2f343e6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.548382 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a6d20-34e4-4533-980a-07c2f343e6b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.549037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a6d20-34e4-4533-980a-07c2f343e6b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:34.550408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25a6d20-34e4-4533-980a-07c2f343e6b5 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-e25a6d20-34e4-4533-980a-07c2f343e6b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:34.562398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:34.563443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:34.564319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 505/988] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:34.569475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a6d20-34e4-4533-980a-07c2f343e6b5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:34.600139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03ca9c-61ef-47f8-aef1-693ada88c7e5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:34.625967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:34.634090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ed28d-a66a-43d6-bfa6-d5e78385234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:34.634790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ed28d-a66a-43d6-bfa6-d5e78385234a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.635101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ed28d-a66a-43d6-bfa6-d5e78385234a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.635382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ed28d-a66a-43d6-bfa6-d5e78385234a None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.636261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ed28d-a66a-43d6-bfa6-d5e78385234a None admin] RBAC: Adding query filter params (name=image) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:34.636375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:34.636458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ed28d-a66a-43d6-bfa6-d5e78385234a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:34.637892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ed28d-a66a-43d6-bfa6-d5e78385234a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:34.642580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 484/989] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:34.643796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a244e3-e61d-440a-bbc5-513fd1aa55c6 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.644086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a244e3-e61d-440a-bbc5-513fd1aa55c6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.644366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a244e3-e61d-440a-bbc5-513fd1aa55c6 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.645579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 485/990] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:34.650575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a244e3-e61d-440a-bbc5-513fd1aa55c6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:34.672116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a6d20-34e4-4533-980a-07c2f343e6b5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:34.705820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff87155-e603-4970-90dc-3d179723699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:34.706494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff87155-e603-4970-90dc-3d179723699e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.706788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff87155-e603-4970-90dc-3d179723699e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.707097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff87155-e603-4970-90dc-3d179723699e None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.707880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff87155-e603-4970-90dc-3d179723699e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:34.708262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff87155-e603-4970-90dc-3d179723699e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:34.709644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff87155-e603-4970-90dc-3d179723699e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:34.714874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 506/991] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 5 15:01:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:34.716096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcbd13-0fdc-461f-bc43-f529eb889972 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.716390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcbd13-0fdc-461f-bc43-f529eb889972 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.716674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcbd13-0fdc-461f-bc43-f529eb889972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.732453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-21dcbd13-0fdc-461f-bc43-f529eb889972 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:34.733580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-21dcbd13-0fdc-461f-bc43-f529eb889972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:34.754621 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a244e3-e61d-440a-bbc5-513fd1aa55c6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:34.788520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cc336-380a-4b2e-b019-32e9ef20ecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:34.789178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cc336-380a-4b2e-b019-32e9ef20ecd3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.789467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cc336-380a-4b2e-b019-32e9ef20ecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.789743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cc336-380a-4b2e-b019-32e9ef20ecd3 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.790452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867cc336-380a-4b2e-b019-32e9ef20ecd3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:34.790763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867cc336-380a-4b2e-b019-32e9ef20ecd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:34.792252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867cc336-380a-4b2e-b019-32e9ef20ecd3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:34.796684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 486/992] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:34 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:34.797441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cc336-380a-4b2e-b019-32e9ef20ecd3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:34.800579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 507/993] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:34 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:34.823870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcbd13-0fdc-461f-bc43-f529eb889972 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:34.873193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:34.892506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:34.926462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867cc336-380a-4b2e-b019-32e9ef20ecd3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:34.938527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcbd13-0fdc-461f-bc43-f529eb889972 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:34.970951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234b265-c0a9-43b4-bc34-8dccfb1dc388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:34.971665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234b265-c0a9-43b4-bc34-8dccfb1dc388 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.972117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234b265-c0a9-43b4-bc34-8dccfb1dc388 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.972402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234b265-c0a9-43b4-bc34-8dccfb1dc388 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.973710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234b265-c0a9-43b4-bc34-8dccfb1dc388 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:34.974036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234b265-c0a9-43b4-bc34-8dccfb1dc388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:34.975482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234b265-c0a9-43b4-bc34-8dccfb1dc388 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:34.978124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95ee38-3fbf-4413-a337-aa7813cf20cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:34.978960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95ee38-3fbf-4413-a337-aa7813cf20cc None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:34.979298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95ee38-3fbf-4413-a337-aa7813cf20cc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:34.979947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c95ee38-3fbf-4413-a337-aa7813cf20cc None admin] PATH_INFO: `/v3/services/compute` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:34.983228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95ee38-3fbf-4413-a337-aa7813cf20cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:34.983354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 487/994] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 5 15:01:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:34.983436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c95ee38-3fbf-4413-a337-aa7813cf20cc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:34.986475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234b265-c0a9-43b4-bc34-8dccfb1dc388 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-6c95ee38-3fbf-4413-a337-aa7813cf20cc None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:34.994922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:34.996100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:34.997072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 508/995] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:34 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:35.005032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95ee38-3fbf-4413-a337-aa7813cf20cc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:35.104047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234b265-c0a9-43b4-bc34-8dccfb1dc388 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:35.112658 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c95ee38-3fbf-4413-a337-aa7813cf20cc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:35.125134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:35.139717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:35.140258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716526ed-b44c-46a9-9d0e-df484455ef41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:35.140995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716526ed-b44c-46a9-9d0e-df484455ef41 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:35.141398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716526ed-b44c-46a9-9d0e-df484455ef41 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:35.141730 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716526ed-b44c-46a9-9d0e-df484455ef41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:35.142707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716526ed-b44c-46a9-9d0e-df484455ef41 None admin] RBAC: Adding query filter params (interface=admin, service_id=b5edce12426d40dea1a4d7fb847407c3, region_id=RegionOne) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:35.143141 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716526ed-b44c-46a9-9d0e-df484455ef41 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:35.144563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716526ed-b44c-46a9-9d0e-df484455ef41 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:35.149782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 488/996] 10.222.0.23 () {56 vars in 1249 bytes} [Tue Jul 5 15:01:34 2022] GET /identity/v3/endpoints?service_id=b5edce12426d40dea1a4d7fb847407c3&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:35.150958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4243939-473d-4dfc-a29c-47d0dd7a2ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:35.151837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4243939-473d-4dfc-a29c-47d0dd7a2ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:35.152324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4243939-473d-4dfc-a29c-47d0dd7a2ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:35.152746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4243939-473d-4dfc-a29c-47d0dd7a2ff6 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:35.153618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4243939-473d-4dfc-a29c-47d0dd7a2ff6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:35.154091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4243939-473d-4dfc-a29c-47d0dd7a2ff6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:35.155665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716526ed-b44c-46a9-9d0e-df484455ef41 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:35.155917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4243939-473d-4dfc-a29c-47d0dd7a2ff6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:35.161371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 509/997] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:35.167442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4243939-473d-4dfc-a29c-47d0dd7a2ff6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:35.266221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716526ed-b44c-46a9-9d0e-df484455ef41 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:35.276973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4243939-473d-4dfc-a29c-47d0dd7a2ff6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:35.302659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75cf00-7d9f-4c30-becb-4f92e1988031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:35.303347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75cf00-7d9f-4c30-becb-4f92e1988031 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:35.303629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75cf00-7d9f-4c30-becb-4f92e1988031 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:35.303907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75cf00-7d9f-4c30-becb-4f92e1988031 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:35.304587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75cf00-7d9f-4c30-becb-4f92e1988031 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:35.305940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75cf00-7d9f-4c30-becb-4f92e1988031 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:35.312383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f352c4-6ce1-48e2-92ce-13f9c0aa7497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:35.313065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f352c4-6ce1-48e2-92ce-13f9c0aa7497 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:35.313357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f352c4-6ce1-48e2-92ce-13f9c0aa7497 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:35.323129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f352c4-6ce1-48e2-92ce-13f9c0aa7497 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:35.323345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 489/998] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 5 15:01:35 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:01:35.323467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f352c4-6ce1-48e2-92ce-13f9c0aa7497 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:35.323467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f352c4-6ce1-48e2-92ce-13f9c0aa7497 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:35.323467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f352c4-6ce1-48e2-92ce-13f9c0aa7497 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:35.329054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 510/999] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:35.334888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75cf00-7d9f-4c30-becb-4f92e1988031 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:35.389836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:35.445451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75cf00-7d9f-4c30-becb-4f92e1988031 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:35.484428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75889625-7332-465f-aa0a-87c29afa8897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:35.485089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75889625-7332-465f-aa0a-87c29afa8897 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:35.485371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75889625-7332-465f-aa0a-87c29afa8897 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:35.485642 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75889625-7332-465f-aa0a-87c29afa8897 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:35.486439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75889625-7332-465f-aa0a-87c29afa8897 None admin] RBAC: Adding query filter params (interface=public, service_id=70a1ce1da5b84cf8a8cdf7aad1014692, region_id=RegionOne) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:35.486753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75889625-7332-465f-aa0a-87c29afa8897 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:35.488400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75889625-7332-465f-aa0a-87c29afa8897 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:35.493670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 490/1000] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 5 15:01:35 2022] GET /identity/v3/endpoints?service_id=70a1ce1da5b84cf8a8cdf7aad1014692&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:35.499886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f352c4-6ce1-48e2-92ce-13f9c0aa7497 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:35.554642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:35.600399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f352c4-6ce1-48e2-92ce-13f9c0aa7497 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:35.632774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2def1b2e-7ad7-4638-b924-98f6b157f293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:35.633441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def1b2e-7ad7-4638-b924-98f6b157f293 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:35.633723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def1b2e-7ad7-4638-b924-98f6b157f293 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:35.633999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2def1b2e-7ad7-4638-b924-98f6b157f293 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:35.635291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def1b2e-7ad7-4638-b924-98f6b157f293 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:35.636662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2def1b2e-7ad7-4638-b924-98f6b157f293 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:35.651089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 511/1001] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 5 15:01:35 2022] GET /identity/v3/services => generated 1727 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:01:36.828242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bd43f-718b-4fc8-b376-e64e479a3062 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:36.828790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bd43f-718b-4fc8-b376-e64e479a3062 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:36.828790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bd43f-718b-4fc8-b376-e64e479a3062 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:36.830303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 491/1002] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:36.835240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d8c4f-f327-4452-8be8-398bf79a9ec1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:36.835678 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d8c4f-f327-4452-8be8-398bf79a9ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:36.835821 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d8c4f-f327-4452-8be8-398bf79a9ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:36.852829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-369d8c4f-f327-4452-8be8-398bf79a9ec1 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:36.854150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-369d8c4f-f327-4452-8be8-398bf79a9ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:36.923086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 512/1003] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:36.943865 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bd43f-718b-4fc8-b376-e64e479a3062 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:37.047920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bd43f-718b-4fc8-b376-e64e479a3062 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:37.062326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:37.084741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c29a5d7-c35e-44b1-bb2f-35c88ba68266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:37.085339 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29a5d7-c35e-44b1-bb2f-35c88ba68266 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:37.085626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29a5d7-c35e-44b1-bb2f-35c88ba68266 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:37.085905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c29a5d7-c35e-44b1-bb2f-35c88ba68266 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:37.086640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29a5d7-c35e-44b1-bb2f-35c88ba68266 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:37.088659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c29a5d7-c35e-44b1-bb2f-35c88ba68266 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:37.102373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 492/1004] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:01:36 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:01:37.358222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48023ce9-a5b5-492e-b276-e7802e73868c None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:37.358408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48023ce9-a5b5-492e-b276-e7802e73868c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:37.358697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48023ce9-a5b5-492e-b276-e7802e73868c None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:37.360123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 513/1005] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:37.365198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14b5fa-ef07-4153-be96-4eb7f6ad1570 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:37.365498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14b5fa-ef07-4153-be96-4eb7f6ad1570 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:37.365783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14b5fa-ef07-4153-be96-4eb7f6ad1570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:37.382960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fc14b5fa-ef07-4153-be96-4eb7f6ad1570 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:37.384169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fc14b5fa-ef07-4153-be96-4eb7f6ad1570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:37.459191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 493/1006] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:37 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:37.481532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48023ce9-a5b5-492e-b276-e7802e73868c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:37.532447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:37.587337 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48023ce9-a5b5-492e-b276-e7802e73868c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:37.621919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6314f3d9-fbd5-48ae-81b3-d19b6c408b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:37.622724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314f3d9-fbd5-48ae-81b3-d19b6c408b15 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:37.623009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314f3d9-fbd5-48ae-81b3-d19b6c408b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:37.623309 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6314f3d9-fbd5-48ae-81b3-d19b6c408b15 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:37.623962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314f3d9-fbd5-48ae-81b3-d19b6c408b15 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:37.625325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6314f3d9-fbd5-48ae-81b3-d19b6c408b15 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-6314f3d9-fbd5-48ae-81b3-d19b6c408b15 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:37.637095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:37.638087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 05 15:01:37.638969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:37.643395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 514/1007] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 5 15:01:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:37.644613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14b5fa-ef07-4153-be96-4eb7f6ad1570 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:37.649758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fca88-a174-4954-bcc5-6d12a88468e0 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:37.650081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fca88-a174-4954-bcc5-6d12a88468e0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:37.650365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6fca88-a174-4954-bcc5-6d12a88468e0 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:37.651921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 515/1008] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:37.656546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35b4e-9064-4e6e-968e-5dd78b6a1561 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:37.656840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35b4e-9064-4e6e-968e-5dd78b6a1561 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:37.657117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b35b4e-9064-4e6e-968e-5dd78b6a1561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:37.674180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c5b35b4e-9064-4e6e-968e-5dd78b6a1561 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:37.675387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c5b35b4e-9064-4e6e-968e-5dd78b6a1561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:37.745223 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 516/1009] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:37 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:37.747204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:37.753971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc14b5fa-ef07-4153-be96-4eb7f6ad1570 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:37.767372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b35b4e-9064-4e6e-968e-5dd78b6a1561 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:37.792867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b19e3-42bb-46c0-89d1-b280f4af86fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:37.793522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b19e3-42bb-46c0-89d1-b280f4af86fd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:37.793802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b19e3-42bb-46c0-89d1-b280f4af86fd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:37.794078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b19e3-42bb-46c0-89d1-b280f4af86fd None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:37.794793 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b19e3-42bb-46c0-89d1-b280f4af86fd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:37.795138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b19e3-42bb-46c0-89d1-b280f4af86fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:37.796518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b19e3-42bb-46c0-89d1-b280f4af86fd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:37.801468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 494/1010] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 5 15:01:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:37.807133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b19e3-42bb-46c0-89d1-b280f4af86fd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:37.817369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:37.876861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b35b4e-9064-4e6e-968e-5dd78b6a1561 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:37.911765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff99f15-0afb-4610-99bf-24b4c485add2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:37.912560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff99f15-0afb-4610-99bf-24b4c485add2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:37.912848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff99f15-0afb-4610-99bf-24b4c485add2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:37.913128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff99f15-0afb-4610-99bf-24b4c485add2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:37.913785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff99f15-0afb-4610-99bf-24b4c485add2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:37.915204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff99f15-0afb-4610-99bf-24b4c485add2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:37.916946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b19e3-42bb-46c0-89d1-b280f4af86fd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-4ff99f15-0afb-4610-99bf-24b4c485add2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:37.929209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:37.930306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:37.932414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 517/1011] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:37.936755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff99f15-0afb-4610-99bf-24b4c485add2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:37.955721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418333b-078e-4787-86df-7f38529b727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:37.956389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418333b-078e-4787-86df-7f38529b727d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:37.956678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418333b-078e-4787-86df-7f38529b727d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:37.956969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418333b-078e-4787-86df-7f38529b727d None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:37.957676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418333b-078e-4787-86df-7f38529b727d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:37.957989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418333b-078e-4787-86df-7f38529b727d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:37.959386 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418333b-078e-4787-86df-7f38529b727d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:37.965394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 495/1012] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 5 15:01:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:37.970857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418333b-078e-4787-86df-7f38529b727d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:38.001670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:38.049585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff99f15-0afb-4610-99bf-24b4c485add2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:38.057758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:38.085400 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ae158-db05-44e1-905f-65ffa3daf55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:38.086071 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae158-db05-44e1-905f-65ffa3daf55f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:38.086359 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae158-db05-44e1-905f-65ffa3daf55f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:38.086640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ae158-db05-44e1-905f-65ffa3daf55f None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:38.086982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418333b-078e-4787-86df-7f38529b727d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:38.087388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ae158-db05-44e1-905f-65ffa3daf55f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:38.088235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ae158-db05-44e1-905f-65ffa3daf55f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:38.089604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ae158-db05-44e1-905f-65ffa3daf55f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:38.095401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 518/1013] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:38.100827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ae158-db05-44e1-905f-65ffa3daf55f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:38.129402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc59bae-5f13-460b-9eac-098e3a995c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:38.130071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc59bae-5f13-460b-9eac-098e3a995c40 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:38.130356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc59bae-5f13-460b-9eac-098e3a995c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:38.130636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc59bae-5f13-460b-9eac-098e3a995c40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:38.131406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc59bae-5f13-460b-9eac-098e3a995c40 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:38.132771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc59bae-5f13-460b-9eac-098e3a995c40 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:38.148433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 496/1014] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 5 15:01:37 2022] POST /identity/v3/endpoints => generated 325 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:38.221466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ae158-db05-44e1-905f-65ffa3daf55f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:38.268590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d288f46-f830-4bea-b206-3d999e5e75d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:38.269663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288f46-f830-4bea-b206-3d999e5e75d2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:38.270162 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288f46-f830-4bea-b206-3d999e5e75d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:38.270644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d288f46-f830-4bea-b206-3d999e5e75d2 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:38.271615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d288f46-f830-4bea-b206-3d999e5e75d2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:38.272168 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d288f46-f830-4bea-b206-3d999e5e75d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:38.273833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d288f46-f830-4bea-b206-3d999e5e75d2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:38.280722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 519/1015] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 5 15:01:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:38.286120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc59bae-5f13-460b-9eac-098e3a995c40 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:38.377632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:38.394941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc59bae-5f13-460b-9eac-098e3a995c40 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:38.428369 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fd6cc-95b6-41d5-b990-7b041b90a276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:38.429030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fd6cc-95b6-41d5-b990-7b041b90a276 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:38.429315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fd6cc-95b6-41d5-b990-7b041b90a276 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:38.429594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fd6cc-95b6-41d5-b990-7b041b90a276 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:38.430349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fd6cc-95b6-41d5-b990-7b041b90a276 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:38.432400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fd6cc-95b6-41d5-b990-7b041b90a276 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:38.459073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 497/1016] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 5 15:01:38 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:39.153448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d0e2-36df-4dc2-a404-77fffa0f54d9 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:39.153970 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d0e2-36df-4dc2-a404-77fffa0f54d9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:39.154081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d0e2-36df-4dc2-a404-77fffa0f54d9 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:39.155566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 520/1017] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:39.160447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96cce3-dd75-4e0b-b064-d422afa46c73 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:39.160836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96cce3-dd75-4e0b-b064-d422afa46c73 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:39.161018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96cce3-dd75-4e0b-b064-d422afa46c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:39.178119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2c96cce3-dd75-4e0b-b064-d422afa46c73 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:39.179314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2c96cce3-dd75-4e0b-b064-d422afa46c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:39.253041 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 498/1018] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:39 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:39.276362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455d0e2-36df-4dc2-a404-77fffa0f54d9 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:39.306491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:39.377742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455d0e2-36df-4dc2-a404-77fffa0f54d9 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:39.410013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fea345-c4ea-44b6-9590-fa1c7e3c5265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:39.410739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea345-c4ea-44b6-9590-fa1c7e3c5265 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:39.411023 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea345-c4ea-44b6-9590-fa1c7e3c5265 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:39.411327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fea345-c4ea-44b6-9590-fa1c7e3c5265 None admin] PATH_INFO: `/v3/services/image` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:39.411976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fea345-c4ea-44b6-9590-fa1c7e3c5265 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:39.413336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fea345-c4ea-44b6-9590-fa1c7e3c5265 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-f9fea345-c4ea-44b6-9590-fa1c7e3c5265 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:39.424679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:39.425722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:01:39.426591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:39.428035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 521/1019] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Jul 5 15:01:39 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:39.432221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96cce3-dd75-4e0b-b064-d422afa46c73 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:39.498856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:39.532064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96cce3-dd75-4e0b-b064-d422afa46c73 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:39.565489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799b31a-60e3-439f-9ba2-b80c53125fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:39.566152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799b31a-60e3-439f-9ba2-b80c53125fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:39.566439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799b31a-60e3-439f-9ba2-b80c53125fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:39.566715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6799b31a-60e3-439f-9ba2-b80c53125fe3 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:39.567448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799b31a-60e3-439f-9ba2-b80c53125fe3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:39.567762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799b31a-60e3-439f-9ba2-b80c53125fe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:39.569128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6799b31a-60e3-439f-9ba2-b80c53125fe3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:39.578857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 499/1020] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:39 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:39.579887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fea345-c4ea-44b6-9590-fa1c7e3c5265 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:39.679073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:39.687784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fea345-c4ea-44b6-9590-fa1c7e3c5265 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:39.721843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecdd38-c7e0-428b-b21d-affbf5996a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:39.722508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecdd38-c7e0-428b-b21d-affbf5996a56 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:39.722797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecdd38-c7e0-428b-b21d-affbf5996a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:39.723112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecdd38-c7e0-428b-b21d-affbf5996a56 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:39.723823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecdd38-c7e0-428b-b21d-affbf5996a56 None admin] RBAC: Adding query filter params (type=image) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:39.724134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecdd38-c7e0-428b-b21d-affbf5996a56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:39.725496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecdd38-c7e0-428b-b21d-affbf5996a56 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:39.735406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 522/1021] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:39 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:39.736359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799b31a-60e3-439f-9ba2-b80c53125fe3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:39.838431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6799b31a-60e3-439f-9ba2-b80c53125fe3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:39.869842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:39.876334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a57556-4a67-41f0-9c44-547b23a10f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:39.877007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a57556-4a67-41f0-9c44-547b23a10f15 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:39.877298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a57556-4a67-41f0-9c44-547b23a10f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:39.877577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a57556-4a67-41f0-9c44-547b23a10f15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:39.878375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a57556-4a67-41f0-9c44-547b23a10f15 None admin] RBAC: Adding query filter params (interface=public, service_id=ad42ab174008469387aec0335c67fc53, region_id=RegionOne) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:39.878694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a57556-4a67-41f0-9c44-547b23a10f15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:39.880212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a57556-4a67-41f0-9c44-547b23a10f15 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:39.887204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 500/1022] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 5 15:01:39 2022] GET /identity/v3/endpoints?service_id=ad42ab174008469387aec0335c67fc53&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:39.891883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecdd38-c7e0-428b-b21d-affbf5996a56 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:40.004377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecdd38-c7e0-428b-b21d-affbf5996a56 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:40.037834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed494b39-2ef1-4522-bb85-9f26a44e5ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:40.038501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed494b39-2ef1-4522-bb85-9f26a44e5ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:40.038786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed494b39-2ef1-4522-bb85-9f26a44e5ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:40.039080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed494b39-2ef1-4522-bb85-9f26a44e5ef2 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:40.039770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed494b39-2ef1-4522-bb85-9f26a44e5ef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:40.041642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed494b39-2ef1-4522-bb85-9f26a44e5ef2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:40.055159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 523/1023] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 5 15:01:39 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:01:40.208268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6595-1abe-4ed6-8689-3404dc4f8826 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:40.208768 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6595-1abe-4ed6-8689-3404dc4f8826 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:40.208892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b6595-1abe-4ed6-8689-3404dc4f8826 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:40.210200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 501/1024] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:40.215893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bac5b-9dff-4c09-a88a-fab9d6f9cd54 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:40.216327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bac5b-9dff-4c09-a88a-fab9d6f9cd54 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:40.216527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6bac5b-9dff-4c09-a88a-fab9d6f9cd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:40.234273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ac6bac5b-9dff-4c09-a88a-fab9d6f9cd54 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:40.235875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ac6bac5b-9dff-4c09-a88a-fab9d6f9cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:40.253752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:40.305332 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 524/1025] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:40.329393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b6595-1abe-4ed6-8689-3404dc4f8826 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:40.439454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b6595-1abe-4ed6-8689-3404dc4f8826 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:40.473751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0560e6a8-ac8d-4a19-8266-246d6570d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:40.474543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560e6a8-ac8d-4a19-8266-246d6570d6df None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:40.474828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560e6a8-ac8d-4a19-8266-246d6570d6df None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:40.475134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0560e6a8-ac8d-4a19-8266-246d6570d6df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:40.476301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560e6a8-ac8d-4a19-8266-246d6570d6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:40.477733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0560e6a8-ac8d-4a19-8266-246d6570d6df None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:40.484786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 502/1026] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:40.486267 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bac5b-9dff-4c09-a88a-fab9d6f9cd54 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:40.585525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6bac5b-9dff-4c09-a88a-fab9d6f9cd54 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:40.597731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ead1d1-7eef-4d90-9902-2ec3dd53de30 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:40.598024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ead1d1-7eef-4d90-9902-2ec3dd53de30 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:40.598304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ead1d1-7eef-4d90-9902-2ec3dd53de30 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:40.599698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 503/1027] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:40.604353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069fab0-ee90-4fbb-b8ee-3a4ac2727599 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:40.604639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069fab0-ee90-4fbb-b8ee-3a4ac2727599 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:40.604917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e069fab0-ee90-4fbb-b8ee-3a4ac2727599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:40.621045 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e069fab0-ee90-4fbb-b8ee-3a4ac2727599 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:40.622216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e069fab0-ee90-4fbb-b8ee-3a4ac2727599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:40.623916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cab762-4517-4c22-9335-8825678019dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:40.624752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cab762-4517-4c22-9335-8825678019dd None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:40.625043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cab762-4517-4c22-9335-8825678019dd None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:40.625325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cab762-4517-4c22-9335-8825678019dd None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:40.626050 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cab762-4517-4c22-9335-8825678019dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:40.627531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cab762-4517-4c22-9335-8825678019dd None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:40.655062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 525/1028] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:01:40 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:40.693470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:40.704252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 504/1029] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:40.727730 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cab762-4517-4c22-9335-8825678019dd None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:40.732293 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:40.838712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cab762-4517-4c22-9335-8825678019dd None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:40.874945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb225d5c-b32a-4e7e-94c0-438cf9ffcd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:40.875809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb225d5c-b32a-4e7e-94c0-438cf9ffcd1d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:40.876091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb225d5c-b32a-4e7e-94c0-438cf9ffcd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:40.876371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb225d5c-b32a-4e7e-94c0-438cf9ffcd1d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:40.886591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb225d5c-b32a-4e7e-94c0-438cf9ffcd1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-cb225d5c-b32a-4e7e-94c0-438cf9ffcd1d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:40.894704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:40.895882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:40.896760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 526/1030] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 5 15:01:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:40.902577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069fab0-ee90-4fbb-b8ee-3a4ac2727599 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:41.001132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e069fab0-ee90-4fbb-b8ee-3a4ac2727599 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:41.034363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed8afc-28d6-4ae7-8009-30ca40a69bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:41.035031 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed8afc-28d6-4ae7-8009-30ca40a69bba None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:41.035337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed8afc-28d6-4ae7-8009-30ca40a69bba None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:41.035612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed8afc-28d6-4ae7-8009-30ca40a69bba None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:41.036358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed8afc-28d6-4ae7-8009-30ca40a69bba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:41.036667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed8afc-28d6-4ae7-8009-30ca40a69bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:41.038030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed8afc-28d6-4ae7-8009-30ca40a69bba None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:41.047084 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 505/1031] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 5 15:01:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:41.052537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb225d5c-b32a-4e7e-94c0-438cf9ffcd1d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:41.114518 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:41.154210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb225d5c-b32a-4e7e-94c0-438cf9ffcd1d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:41.190950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1fe52-39b3-4659-82c5-72a1e45ca2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:41.194387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1fe52-39b3-4659-82c5-72a1e45ca2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:41.194387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1fe52-39b3-4659-82c5-72a1e45ca2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:41.194387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1fe52-39b3-4659-82c5-72a1e45ca2b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:41.207405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1fe52-39b3-4659-82c5-72a1e45ca2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:41.208874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1fe52-39b3-4659-82c5-72a1e45ca2b5 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-bec1fe52-39b3-4659-82c5-72a1e45ca2b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:41.226410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:41.233131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:41.234122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:41.234122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:41.234122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:41.234122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:41.234122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 527/1032] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:41.234425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed8afc-28d6-4ae7-8009-30ca40a69bba None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:41.289531 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:41.334378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed8afc-28d6-4ae7-8009-30ca40a69bba None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:41.367584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5de3c2-9593-4f30-8c22-f3749cb32b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:41.368171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5de3c2-9593-4f30-8c22-f3749cb32b44 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:41.368454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5de3c2-9593-4f30-8c22-f3749cb32b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:41.368755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5de3c2-9593-4f30-8c22-f3749cb32b44 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:41.369470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5de3c2-9593-4f30-8c22-f3749cb32b44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:41.369774 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5de3c2-9593-4f30-8c22-f3749cb32b44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:41.371155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5de3c2-9593-4f30-8c22-f3749cb32b44 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:41.382686 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 506/1033] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:41.389366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1fe52-39b3-4659-82c5-72a1e45ca2b5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:41.487345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1fe52-39b3-4659-82c5-72a1e45ca2b5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:41.502838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:41.521376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8d005-3b80-48ed-b188-0c87e5619d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:41.522167 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8d005-3b80-48ed-b188-0c87e5619d6c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:41.522463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8d005-3b80-48ed-b188-0c87e5619d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:41.522729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8d005-3b80-48ed-b188-0c87e5619d6c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:41.530817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8d005-3b80-48ed-b188-0c87e5619d6c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:41.532291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8d005-3b80-48ed-b188-0c87e5619d6c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-97a8d005-3b80-48ed-b188-0c87e5619d6c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:41.546089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:41.547035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:41.548304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:41.548304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:41.548304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:41.548304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:41.548304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:41.548304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:41.548304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:41.548304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:41.548304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:41.548304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:41.549294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 528/1034] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 5 15:01:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:41.552868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5de3c2-9593-4f30-8c22-f3749cb32b44 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:41.653170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5de3c2-9593-4f30-8c22-f3749cb32b44 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:41.669561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:41.688980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d3012-bea5-4b0f-bbec-ccbfcdfa8931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:41.689704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d3012-bea5-4b0f-bbec-ccbfcdfa8931 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:41.689994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d3012-bea5-4b0f-bbec-ccbfcdfa8931 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:41.690271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d3012-bea5-4b0f-bbec-ccbfcdfa8931 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:41.691139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d3012-bea5-4b0f-bbec-ccbfcdfa8931 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:41.691456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d3012-bea5-4b0f-bbec-ccbfcdfa8931 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:41.692897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d3012-bea5-4b0f-bbec-ccbfcdfa8931 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:41.715138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 507/1035] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 5 15:01:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:41.716698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8d005-3b80-48ed-b188-0c87e5619d6c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:41.823546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8d005-3b80-48ed-b188-0c87e5619d6c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:41.857467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235f686-240d-4687-8da4-fd4cd16c3c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:41.858122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235f686-240d-4687-8da4-fd4cd16c3c03 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:41.858413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235f686-240d-4687-8da4-fd4cd16c3c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:41.858695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235f686-240d-4687-8da4-fd4cd16c3c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:41.905887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:41.929482 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235f686-240d-4687-8da4-fd4cd16c3c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:41.930942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235f686-240d-4687-8da4-fd4cd16c3c03 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:42.033317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 529/1036] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:41 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:01:42.035018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d3012-bea5-4b0f-bbec-ccbfcdfa8931 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:42.124473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be886161-bf2d-4057-a8dc-9d4a086cacd9 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.124921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be886161-bf2d-4057-a8dc-9d4a086cacd9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.125287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be886161-bf2d-4057-a8dc-9d4a086cacd9 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.126795 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 530/1037] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:01:42.131499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f303f-9803-4367-8b9d-c016f8cd3de1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.131806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f303f-9803-4367-8b9d-c016f8cd3de1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.132082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f303f-9803-4367-8b9d-c016f8cd3de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.137816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d3012-bea5-4b0f-bbec-ccbfcdfa8931 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:42.148664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-175f303f-9803-4367-8b9d-c016f8cd3de1 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:42.149962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-175f303f-9803-4367-8b9d-c016f8cd3de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:42.174500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016558f-26f9-4528-b0b9-faa10af7cc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:42.175252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016558f-26f9-4528-b0b9-faa10af7cc77 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.175540 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016558f-26f9-4528-b0b9-faa10af7cc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.175826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7016558f-26f9-4528-b0b9-faa10af7cc77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.186002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016558f-26f9-4528-b0b9-faa10af7cc77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:42.187555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7016558f-26f9-4528-b0b9-faa10af7cc77 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-7016558f-26f9-4528-b0b9-faa10af7cc77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:42.212693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:42.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:42.219470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:42.219470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:42.219470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:42.219470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:42.219470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 508/1038] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:42.220691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016558f-26f9-4528-b0b9-faa10af7cc77 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:42.227932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 531/1039] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:42.249204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:42.251326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f303f-9803-4367-8b9d-c016f8cd3de1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:42.260668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:42.330522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7016558f-26f9-4528-b0b9-faa10af7cc77 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:42.361230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f303f-9803-4367-8b9d-c016f8cd3de1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:42.367340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e9f10-7067-490e-8f64-dcc1d1334937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:42.368016 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e9f10-7067-490e-8f64-dcc1d1334937 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.368308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e9f10-7067-490e-8f64-dcc1d1334937 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.368619 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e9f10-7067-490e-8f64-dcc1d1334937 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.369519 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e9f10-7067-490e-8f64-dcc1d1334937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:42.369780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e9f10-7067-490e-8f64-dcc1d1334937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:42.371190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e9f10-7067-490e-8f64-dcc1d1334937 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:42.387168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 509/1040] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:42.388226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e9f10-7067-490e-8f64-dcc1d1334937 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:42.404447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7873c5-0c6b-4a56-aada-3980e191d885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:42.412302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7873c5-0c6b-4a56-aada-3980e191d885 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.412302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7873c5-0c6b-4a56-aada-3980e191d885 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.412302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7873c5-0c6b-4a56-aada-3980e191d885 None admin] PATH_INFO: `/v3/services/image` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.412302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7873c5-0c6b-4a56-aada-3980e191d885 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:42.412302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7873c5-0c6b-4a56-aada-3980e191d885 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-ae7873c5-0c6b-4a56-aada-3980e191d885 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:42.424285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:42.425605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 05 15:01:42.426735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:42.430655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 532/1041] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Jul 5 15:01:42 2022] GET /identity/v3/services/image => generated 86 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:42.432516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7873c5-0c6b-4a56-aada-3980e191d885 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:42.488626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:42.504649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e9f10-7067-490e-8f64-dcc1d1334937 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:42.509083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:42.542710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7873c5-0c6b-4a56-aada-3980e191d885 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:42.545698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ce4e-26ae-4d36-abfb-0b4f8f3b2eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:42.545966 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ce4e-26ae-4d36-abfb-0b4f8f3b2eaa None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.546249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ce4e-26ae-4d36-abfb-0b4f8f3b2eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.546530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74ce4e-26ae-4d36-abfb-0b4f8f3b2eaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.553501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74ce4e-26ae-4d36-abfb-0b4f8f3b2eaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:42.554941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74ce4e-26ae-4d36-abfb-0b4f8f3b2eaa None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0b74ce4e-26ae-4d36-abfb-0b4f8f3b2eaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:42.570825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:42.571904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:42.573454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:42.573454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:42.573454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:42.573454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:42.573454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:42.573454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:42.573454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:42.573454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:42.573454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 510/1042] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:42.578174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ce4e-26ae-4d36-abfb-0b4f8f3b2eaa None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:42.581708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fccb38-fd12-4736-8d6c-692993d0cf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:42.582383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fccb38-fd12-4736-8d6c-692993d0cf44 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.582677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fccb38-fd12-4736-8d6c-692993d0cf44 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.582962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fccb38-fd12-4736-8d6c-692993d0cf44 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.583733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fccb38-fd12-4736-8d6c-692993d0cf44 None admin] RBAC: Adding query filter params (name=image) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:42.584052 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fccb38-fd12-4736-8d6c-692993d0cf44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:42.585438 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fccb38-fd12-4736-8d6c-692993d0cf44 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:42.591085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 533/1043] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:42 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:42.596648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fccb38-fd12-4736-8d6c-692993d0cf44 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:42.690344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74ce4e-26ae-4d36-abfb-0b4f8f3b2eaa None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:42.707062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fccb38-fd12-4736-8d6c-692993d0cf44 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:42.729487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccdfa5-728c-44b9-963d-7041d88ddf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:42.730428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccdfa5-728c-44b9-963d-7041d88ddf85 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.730714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccdfa5-728c-44b9-963d-7041d88ddf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.731005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccdfa5-728c-44b9-963d-7041d88ddf85 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.731813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccdfa5-728c-44b9-963d-7041d88ddf85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:42.732138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccdfa5-728c-44b9-963d-7041d88ddf85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:42.733564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccdfa5-728c-44b9-963d-7041d88ddf85 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:42.745235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 511/1044] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:42.745654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348875b-5874-4cea-9423-46255c32a38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:42.746405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348875b-5874-4cea-9423-46255c32a38b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.746697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348875b-5874-4cea-9423-46255c32a38b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.746976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348875b-5874-4cea-9423-46255c32a38b None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.747856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348875b-5874-4cea-9423-46255c32a38b None admin] RBAC: Adding query filter params (type=image) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:42.748171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348875b-5874-4cea-9423-46255c32a38b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:42.749542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2348875b-5874-4cea-9423-46255c32a38b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:42.750999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccdfa5-728c-44b9-963d-7041d88ddf85 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:42.751227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:42.760689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:42.762726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 534/1045] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:42.763928 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348875b-5874-4cea-9423-46255c32a38b None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:42.853382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccdfa5-728c-44b9-963d-7041d88ddf85 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:42.863805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348875b-5874-4cea-9423-46255c32a38b None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:42.886070 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30649e8b-b58d-4c7e-af8d-fca4202124ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:42.886725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30649e8b-b58d-4c7e-af8d-fca4202124ff None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.887008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30649e8b-b58d-4c7e-af8d-fca4202124ff None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.887316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30649e8b-b58d-4c7e-af8d-fca4202124ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.888146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30649e8b-b58d-4c7e-af8d-fca4202124ff None admin] RBAC: Adding query filter params (role.id=e08bb5ad422e4f6593bcbaca2bdf4366, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=e7a416f22d33450183801cef2b13822f) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:42.888463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30649e8b-b58d-4c7e-af8d-fca4202124ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:42.889886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30649e8b-b58d-4c7e-af8d-fca4202124ff None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:42.895171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 512/1046] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:42 2022] GET /identity/v3/role_assignments?user.id=e7a416f22d33450183801cef2b13822f&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=e08bb5ad422e4f6593bcbaca2bdf4366 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:42.896357 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ff4ee-a2bb-4f06-82b9-836407edcaa7 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.896649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ff4ee-a2bb-4f06-82b9-836407edcaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.896921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc7ade-6745-496b-8e26-9dba16341f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:42.897049 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ff4ee-a2bb-4f06-82b9-836407edcaa7 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.897639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc7ade-6745-496b-8e26-9dba16341f20 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.897934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc7ade-6745-496b-8e26-9dba16341f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.899085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defc7ade-6745-496b-8e26-9dba16341f20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.899235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 513/1047] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:42.899326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc7ade-6745-496b-8e26-9dba16341f20 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:42.900487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defc7ade-6745-496b-8e26-9dba16341f20 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:42.904243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0970c-4588-47df-820b-1acec76cdee7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:42.904812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0970c-4588-47df-820b-1acec76cdee7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:42.905283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac0970c-4588-47df-820b-1acec76cdee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:42.924941 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 535/1048] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 5 15:01:42 2022] POST /identity/v3/endpoints => generated 327 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:42.926524 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-6ac0970c-4588-47df-820b-1acec76cdee7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:42.927809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6ac0970c-4588-47df-820b-1acec76cdee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:43.001416 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:43.013928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 514/1049] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:43.038723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc7ade-6745-496b-8e26-9dba16341f20 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:43.107329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:43.156667 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defc7ade-6745-496b-8e26-9dba16341f20 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:43.193964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0fe5e-2004-4a13-a7f7-263e526a2077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:43.193964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0fe5e-2004-4a13-a7f7-263e526a2077 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:43.193964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0fe5e-2004-4a13-a7f7-263e526a2077 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:43.193964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0fe5e-2004-4a13-a7f7-263e526a2077 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:43.194414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0fe5e-2004-4a13-a7f7-263e526a2077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:43.194963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0fe5e-2004-4a13-a7f7-263e526a2077 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:43.197724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 536/1050] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:43.205067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0970c-4588-47df-820b-1acec76cdee7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:43.305921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac0970c-4588-47df-820b-1acec76cdee7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:43.339893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f954fda-53c2-48d8-b740-028150d6dbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:43.340567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f954fda-53c2-48d8-b740-028150d6dbe1 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:43.340855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f954fda-53c2-48d8-b740-028150d6dbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:43.341130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f954fda-53c2-48d8-b740-028150d6dbe1 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:43.341846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f954fda-53c2-48d8-b740-028150d6dbe1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:43.343316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f954fda-53c2-48d8-b740-028150d6dbe1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:43.371301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 515/1051] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:01:43 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:01:44.967983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9488e-bfee-412c-920c-922178d58590 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:44.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9488e-bfee-412c-920c-922178d58590 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:44.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9488e-bfee-412c-920c-922178d58590 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:44.969969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 537/1052] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:44.973737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c022ca-91f7-4906-a81f-5d627852e49a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:44.974211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c022ca-91f7-4906-a81f-5d627852e49a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:44.974299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c022ca-91f7-4906-a81f-5d627852e49a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:44.974620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe30e5-4bd6-4400-aa40-d5b1c4657cda None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:44.974915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe30e5-4bd6-4400-aa40-d5b1c4657cda None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:44.975235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe30e5-4bd6-4400-aa40-d5b1c4657cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:44.975698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 516/1053] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:44.980899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319217cf-ee02-41fc-b2cc-0ac7c3159c72 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:44.981187 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319217cf-ee02-41fc-b2cc-0ac7c3159c72 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:44.981467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319217cf-ee02-41fc-b2cc-0ac7c3159c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:44.992479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-a9fe30e5-4bd6-4400-aa40-d5b1c4657cda None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:44.993719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a9fe30e5-4bd6-4400-aa40-d5b1c4657cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:44.997748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-319217cf-ee02-41fc-b2cc-0ac7c3159c72 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:44.999124 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-319217cf-ee02-41fc-b2cc-0ac7c3159c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:45.050231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:45.061977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 538/1054] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:45.073441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 517/1055] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:45.084828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe30e5-4bd6-4400-aa40-d5b1c4657cda None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:45.098223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319217cf-ee02-41fc-b2cc-0ac7c3159c72 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:45.144347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:45.192643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe30e5-4bd6-4400-aa40-d5b1c4657cda None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:45.216326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319217cf-ee02-41fc-b2cc-0ac7c3159c72 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:45.236425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63f1d2-25ea-4cf3-b8ce-5a53206eb60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:45.237167 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63f1d2-25ea-4cf3-b8ce-5a53206eb60f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.237861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63f1d2-25ea-4cf3-b8ce-5a53206eb60f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.238148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63f1d2-25ea-4cf3-b8ce-5a53206eb60f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:45.248380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63f1d2-25ea-4cf3-b8ce-5a53206eb60f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:45.252012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48e093-fb99-4a1b-bbe3-c4f232e07d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:45.252667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e093-fb99-4a1b-bbe3-c4f232e07d4e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.252956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e093-fb99-4a1b-bbe3-c4f232e07d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.253239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48e093-fb99-4a1b-bbe3-c4f232e07d4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-cb63f1d2-25ea-4cf3-b8ce-5a53206eb60f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:45.256408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:45.257342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:45.258218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 539/1056] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 5 15:01:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:45.264148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63f1d2-25ea-4cf3-b8ce-5a53206eb60f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:45.308620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:45.330319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48e093-fb99-4a1b-bbe3-c4f232e07d4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:45.331825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48e093-fb99-4a1b-bbe3-c4f232e07d4e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:45.369473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63f1d2-25ea-4cf3-b8ce-5a53206eb60f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:45.382267 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:45.404356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fb836-fdb1-4830-90be-f5b74fe75e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:45.405030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fb836-fdb1-4830-90be-f5b74fe75e14 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.405317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fb836-fdb1-4830-90be-f5b74fe75e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.405600 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38fb836-fdb1-4830-90be-f5b74fe75e14 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:45.406376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fb836-fdb1-4830-90be-f5b74fe75e14 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:45.406669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fb836-fdb1-4830-90be-f5b74fe75e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:45.408088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38fb836-fdb1-4830-90be-f5b74fe75e14 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:45.416581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 540/1057] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 5 15:01:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:45.422270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fb836-fdb1-4830-90be-f5b74fe75e14 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:45.446378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 518/1058] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:01:45.448419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264e3f-94b7-4724-a3a6-c08745b07cb9 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.448725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264e3f-94b7-4724-a3a6-c08745b07cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.449012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b264e3f-94b7-4724-a3a6-c08745b07cb9 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:45.450645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 519/1059] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:45.453361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b264e3f-94b7-4724-a3a6-c08745b07cb9 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:45.527555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38fb836-fdb1-4830-90be-f5b74fe75e14 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:45.534398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:45.554746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b264e3f-94b7-4724-a3a6-c08745b07cb9 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:45.560444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcc575-a600-466a-b54c-228e70691a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:45.561240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcc575-a600-466a-b54c-228e70691a03 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.561526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcc575-a600-466a-b54c-228e70691a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.561807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcc575-a600-466a-b54c-228e70691a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:45.572815 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcc575-a600-466a-b54c-228e70691a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:45.574270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcc575-a600-466a-b54c-228e70691a03 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:45.590111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fb680-fb28-4324-a6c8-d55cda44d63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:45.590890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fb680-fb28-4324-a6c8-d55cda44d63e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.591220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fb680-fb28-4324-a6c8-d55cda44d63e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-a0fcc575-a600-466a-b54c-228e70691a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:45.591327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:45.592403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:45.593536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:45.593536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:45.593536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:45.593536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:45.593832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fb680-fb28-4324-a6c8-d55cda44d63e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:45.593925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 541/1060] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:45.593925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71569c7-2b92-45b5-8c84-3559c59498d2 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.593925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71569c7-2b92-45b5-8c84-3559c59498d2 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.594168 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71569c7-2b92-45b5-8c84-3559c59498d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:45.604397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fb680-fb28-4324-a6c8-d55cda44d63e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:45.605842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fb680-fb28-4324-a6c8-d55cda44d63e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:45.609800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b71569c7-2b92-45b5-8c84-3559c59498d2 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:45.610978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b71569c7-2b92-45b5-8c84-3559c59498d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c63fb680-fb28-4324-a6c8-d55cda44d63e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:45.623394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:45.624496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:45.625629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:45.625629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:45.625629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:45.625629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:45.627189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 520/1061] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:45.628164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fb680-fb28-4324-a6c8-d55cda44d63e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:45.631802 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:45.678832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 542/1062] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:45.679862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71569c7-2b92-45b5-8c84-3559c59498d2 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:45.727059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fb680-fb28-4324-a6c8-d55cda44d63e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:45.763719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b37c3-fe80-492c-a3a3-1089ad119f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:45.764401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b37c3-fe80-492c-a3a3-1089ad119f4b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.764686 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b37c3-fe80-492c-a3a3-1089ad119f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.764965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059b37c3-fe80-492c-a3a3-1089ad119f4b None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:45.765746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b37c3-fe80-492c-a3a3-1089ad119f4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:45.766059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b37c3-fe80-492c-a3a3-1089ad119f4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:45.767475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059b37c3-fe80-492c-a3a3-1089ad119f4b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:45.768940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:45.780083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 521/1063] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:45.780811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b37c3-fe80-492c-a3a3-1089ad119f4b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:45.783027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71569c7-2b92-45b5-8c84-3559c59498d2 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:45.817088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c37d31-148e-4f31-8b71-5f2b5bf0aaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:45.817766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c37d31-148e-4f31-8b71-5f2b5bf0aaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.818053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c37d31-148e-4f31-8b71-5f2b5bf0aaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.818334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c37d31-148e-4f31-8b71-5f2b5bf0aaf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:45.819120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c37d31-148e-4f31-8b71-5f2b5bf0aaf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:45.819433 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c37d31-148e-4f31-8b71-5f2b5bf0aaf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:45.820814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c37d31-148e-4f31-8b71-5f2b5bf0aaf6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:45.832814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 543/1064] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:45.833777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c37d31-148e-4f31-8b71-5f2b5bf0aaf6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:45.852670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:45.910845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059b37c3-fe80-492c-a3a3-1089ad119f4b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:45.954573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c37d31-148e-4f31-8b71-5f2b5bf0aaf6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:45.957096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb1d5a-d0ff-4aef-a1b5-cfce9d07fa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:45.958862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb1d5a-d0ff-4aef-a1b5-cfce9d07fa36 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.959183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb1d5a-d0ff-4aef-a1b5-cfce9d07fa36 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.959471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb1d5a-d0ff-4aef-a1b5-cfce9d07fa36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:45.973886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb1d5a-d0ff-4aef-a1b5-cfce9d07fa36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-7ddb1d5a-d0ff-4aef-a1b5-cfce9d07fa36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:45.983781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:45.984985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:45.985981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 522/1065] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:45.990947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb1d5a-d0ff-4aef-a1b5-cfce9d07fa36 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:45.996223 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836db528-d670-47ee-9dd5-a4420cae2642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:45.997575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836db528-d670-47ee-9dd5-a4420cae2642 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:45.998017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836db528-d670-47ee-9dd5-a4420cae2642 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:45.998459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836db528-d670-47ee-9dd5-a4420cae2642 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.007328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836db528-d670-47ee-9dd5-a4420cae2642 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.008801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836db528-d670-47ee-9dd5-a4420cae2642 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-836db528-d670-47ee-9dd5-a4420cae2642 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:46.024510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:46.026097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:46.027116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:46.027116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:46.027116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:46.027116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:46.027116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:46.027116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:46.027116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:46.027116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:46.027116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:46.027116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:46.031259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 544/1066] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 5 15:01:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:46.032798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836db528-d670-47ee-9dd5-a4420cae2642 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:46.058757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:46.112993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb1d5a-d0ff-4aef-a1b5-cfce9d07fa36 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:46.125005 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:46.147865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836db528-d670-47ee-9dd5-a4420cae2642 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:46.150075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b340d50-badc-497f-a025-ea865048ecc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:46.150746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b340d50-badc-497f-a025-ea865048ecc6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:46.151033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b340d50-badc-497f-a025-ea865048ecc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:46.151339 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b340d50-badc-497f-a025-ea865048ecc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.152091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b340d50-badc-497f-a025-ea865048ecc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:46.152403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b340d50-badc-497f-a025-ea865048ecc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.153770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b340d50-badc-497f-a025-ea865048ecc6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.161700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 523/1067] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:01:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:46.167325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b340d50-badc-497f-a025-ea865048ecc6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:46.180584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2873eac-cb56-46a9-9084-cd184755a407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:46.181761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2873eac-cb56-46a9-9084-cd184755a407 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:46.182049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2873eac-cb56-46a9-9084-cd184755a407 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:46.182325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2873eac-cb56-46a9-9084-cd184755a407 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.183194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2873eac-cb56-46a9-9084-cd184755a407 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:46.183508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2873eac-cb56-46a9-9084-cd184755a407 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.184949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2873eac-cb56-46a9-9084-cd184755a407 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.203248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 545/1068] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 5 15:01:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:46.208837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2873eac-cb56-46a9-9084-cd184755a407 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:46.268443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b340d50-badc-497f-a025-ea865048ecc6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:46.291985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:46.302743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0c70d-9a28-4aed-9dae-a691a08c2ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:46.303653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0c70d-9a28-4aed-9dae-a691a08c2ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:46.303938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0c70d-9a28-4aed-9dae-a691a08c2ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:46.304217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0c70d-9a28-4aed-9dae-a691a08c2ae6 None admin] PATH_INFO: `/v3/users/566278d9375f4af3ae6324f800ca7a0c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.308777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2873eac-cb56-46a9-9084-cd184755a407 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:46.317934 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0c70d-9a28-4aed-9dae-a691a08c2ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=566278d9375f4af3ae6324f800ca7a0c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.323061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda0c70d-9a28-4aed-9dae-a691a08c2ae6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.323061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 524/1069] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:01:46 2022] GET /identity/v3/users/566278d9375f4af3ae6324f800ca7a0c => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:46.327712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0c70d-9a28-4aed-9dae-a691a08c2ae6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:46.352295 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c34da-8562-4481-b09b-e3511d5584bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:46.353123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c34da-8562-4481-b09b-e3511d5584bc None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:46.353411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c34da-8562-4481-b09b-e3511d5584bc None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:46.353692 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c34da-8562-4481-b09b-e3511d5584bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.355651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:46.366507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c34da-8562-4481-b09b-e3511d5584bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.369008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c34da-8562-4481-b09b-e3511d5584bc None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-3a8c34da-8562-4481-b09b-e3511d5584bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:46.392691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:46.393688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:46.399738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:46.399738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:46.399738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:46.399738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:46.399738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 546/1070] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:46.400216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c34da-8562-4481-b09b-e3511d5584bc None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:46.429097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0c70d-9a28-4aed-9dae-a691a08c2ae6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:46.465448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b952ff-24f5-42ff-bb98-7dc09d6acac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:46.466091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b952ff-24f5-42ff-bb98-7dc09d6acac5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:46.466370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b952ff-24f5-42ff-bb98-7dc09d6acac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:46.466648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b952ff-24f5-42ff-bb98-7dc09d6acac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.513445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:46.514637 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c34da-8562-4481-b09b-e3511d5584bc None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:46.542676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b952ff-24f5-42ff-bb98-7dc09d6acac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.544081 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b952ff-24f5-42ff-bb98-7dc09d6acac5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.551011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc92e5-8781-48e5-858f-9e82b27bc8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:46.551786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc92e5-8781-48e5-858f-9e82b27bc8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:46.552150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc92e5-8781-48e5-858f-9e82b27bc8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:46.552503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc92e5-8781-48e5-858f-9e82b27bc8c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.553307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc92e5-8781-48e5-858f-9e82b27bc8c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:46.553618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc92e5-8781-48e5-858f-9e82b27bc8c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.554983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc92e5-8781-48e5-858f-9e82b27bc8c9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.568908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 547/1071] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:46.575022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc92e5-8781-48e5-858f-9e82b27bc8c9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:46.621990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:46.649763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 525/1072] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:01:46.655482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b952ff-24f5-42ff-bb98-7dc09d6acac5 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:46.674218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc92e5-8781-48e5-858f-9e82b27bc8c9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:46.707375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c06b6-c385-428b-ab1b-b90a26deb9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:46.708161 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c06b6-c385-428b-ab1b-b90a26deb9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:46.708447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c06b6-c385-428b-ab1b-b90a26deb9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:46.708723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c06b6-c385-428b-ab1b-b90a26deb9e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.715939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c06b6-c385-428b-ab1b-b90a26deb9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.717370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c06b6-c385-428b-ab1b-b90a26deb9e9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.725960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-767c06b6-c385-428b-ab1b-b90a26deb9e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:46.731500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:46.732579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:46.733486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:46.733486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:46.733486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:46.733486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:46.733486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:46.733486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:46.733486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:46.733486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:46.733486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 548/1073] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:46.738234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c06b6-c385-428b-ab1b-b90a26deb9e9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:46.751797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b952ff-24f5-42ff-bb98-7dc09d6acac5 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:46.785353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714ab87-3883-4459-a07e-09a495753c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:46.786200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714ab87-3883-4459-a07e-09a495753c07 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:46.786485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714ab87-3883-4459-a07e-09a495753c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:46.786760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0714ab87-3883-4459-a07e-09a495753c07 None admin] PATH_INFO: `/v3/projects/57f118ac394a4d54b942f4e166623230` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.787964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714ab87-3883-4459-a07e-09a495753c07 None admin] RBAC: Authorizing `identity:get_project(project_id=57f118ac394a4d54b942f4e166623230)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.789394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0714ab87-3883-4459-a07e-09a495753c07 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.791613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 526/1074] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:01:46 2022] GET /identity/v3/projects/57f118ac394a4d54b942f4e166623230 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:46.797532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714ab87-3883-4459-a07e-09a495753c07 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:46.835516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c06b6-c385-428b-ab1b-b90a26deb9e9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:46.843202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:46.870202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7ff80-337f-4b96-b7cf-9a13f6b0cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:46.870863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7ff80-337f-4b96-b7cf-9a13f6b0cf88 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:46.871191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7ff80-337f-4b96-b7cf-9a13f6b0cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:46.871479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7ff80-337f-4b96-b7cf-9a13f6b0cf88 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.872283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7ff80-337f-4b96-b7cf-9a13f6b0cf88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:46.872600 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7ff80-337f-4b96-b7cf-9a13f6b0cf88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.874044 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7ff80-337f-4b96-b7cf-9a13f6b0cf88 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.886037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 549/1075] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:46.891750 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7ff80-337f-4b96-b7cf-9a13f6b0cf88 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:46.893816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0714ab87-3883-4459-a07e-09a495753c07 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:46.925844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ffbda-a6e1-4809-b528-fa36d547c44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:46.926491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ffbda-a6e1-4809-b528-fa36d547c44d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:46.926778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ffbda-a6e1-4809-b528-fa36d547c44d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:46.927077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ffbda-a6e1-4809-b528-fa36d547c44d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:46.927913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ffbda-a6e1-4809-b528-fa36d547c44d None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=57f118ac394a4d54b942f4e166623230, user.id=566278d9375f4af3ae6324f800ca7a0c) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:46.928232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ffbda-a6e1-4809-b528-fa36d547c44d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:46.929722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ffbda-a6e1-4809-b528-fa36d547c44d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:46.935803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 527/1076] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:46 2022] GET /identity/v3/role_assignments?user.id=566278d9375f4af3ae6324f800ca7a0c&scope.project.id=57f118ac394a4d54b942f4e166623230&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:46.991378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7ff80-337f-4b96-b7cf-9a13f6b0cf88 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:47.027305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c264e-8436-4747-91c0-be230d21adc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:47.028219 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c264e-8436-4747-91c0-be230d21adc1 None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:47.028509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c264e-8436-4747-91c0-be230d21adc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:47.028793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c264e-8436-4747-91c0-be230d21adc1 None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/e7a416f22d33450183801cef2b13822f/roles/e08bb5ad422e4f6593bcbaca2bdf4366` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:47.031112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c264e-8436-4747-91c0-be230d21adc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=e7a416f22d33450183801cef2b13822f, role_id=e08bb5ad422e4f6593bcbaca2bdf4366)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:47.032585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c264e-8436-4747-91c0-be230d21adc1 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:47.047487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 550/1077] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:01:46 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/e7a416f22d33450183801cef2b13822f/roles/e08bb5ad422e4f6593bcbaca2bdf4366 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:01:48.913467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6493e38-66fe-43fa-978e-d7766eac4669 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:48.913994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6493e38-66fe-43fa-978e-d7766eac4669 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:48.914122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6493e38-66fe-43fa-978e-d7766eac4669 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:48.915953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 528/1078] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:48.921402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91823757-cad1-47e4-a5a2-4a2a65e3f4ea None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:48.921807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91823757-cad1-47e4-a5a2-4a2a65e3f4ea None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:48.922091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91823757-cad1-47e4-a5a2-4a2a65e3f4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:48.939905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-91823757-cad1-47e4-a5a2-4a2a65e3f4ea None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:48.941073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-91823757-cad1-47e4-a5a2-4a2a65e3f4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:48.960710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:49.006729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 551/1079] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:49.029131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6493e38-66fe-43fa-978e-d7766eac4669 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:49.031341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686125b-2f0f-415e-ae41-2a6cb101ba2b None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.031639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686125b-2f0f-415e-ae41-2a6cb101ba2b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.031926 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686125b-2f0f-415e-ae41-2a6cb101ba2b None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.033304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 552/1080] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:49.038196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cf298-0bc9-474d-a8d4-715171109f91 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.038492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cf298-0bc9-474d-a8d4-715171109f91 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.038775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cf298-0bc9-474d-a8d4-715171109f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.039230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:49.055454 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-6a2cf298-0bc9-474d-a8d4-715171109f91 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:49.057210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6a2cf298-0bc9-474d-a8d4-715171109f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:49.121578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 553/1081] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:01:49.131204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6493e38-66fe-43fa-978e-d7766eac4669 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:49.144560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cf298-0bc9-474d-a8d4-715171109f91 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:49.164491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb2eba-d13c-48de-b7e4-e7202c5b5673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:49.165266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb2eba-d13c-48de-b7e4-e7202c5b5673 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.165549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb2eba-d13c-48de-b7e4-e7202c5b5673 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.165821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb2eba-d13c-48de-b7e4-e7202c5b5673 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.175502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb2eba-d13c-48de-b7e4-e7202c5b5673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d9eb2eba-d13c-48de-b7e4-e7202c5b5673 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:49.183527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:49.184532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:49.185380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 529/1082] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:49.190686 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb2eba-d13c-48de-b7e4-e7202c5b5673 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:49.230829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:49.245917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cf298-0bc9-474d-a8d4-715171109f91 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:49.255458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:49.280040 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd005f7-afd7-49a6-bc58-24bf853ec50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:49.280858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd005f7-afd7-49a6-bc58-24bf853ec50a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.281152 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd005f7-afd7-49a6-bc58-24bf853ec50a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.281437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd005f7-afd7-49a6-bc58-24bf853ec50a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.289360 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb2eba-d13c-48de-b7e4-e7202c5b5673 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:49.291564 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd005f7-afd7-49a6-bc58-24bf853ec50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-4cd005f7-afd7-49a6-bc58-24bf853ec50a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:49.299623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:49.300689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:49.301578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 554/1083] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 5 15:01:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:49.307149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd005f7-afd7-49a6-bc58-24bf853ec50a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:49.323023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a57bf1-39a7-4c34-8f7f-fd023574aacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:49.323715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a57bf1-39a7-4c34-8f7f-fd023574aacc None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.323994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a57bf1-39a7-4c34-8f7f-fd023574aacc None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.324268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a57bf1-39a7-4c34-8f7f-fd023574aacc None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.325014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a57bf1-39a7-4c34-8f7f-fd023574aacc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:49.325573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a57bf1-39a7-4c34-8f7f-fd023574aacc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:49.326923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a57bf1-39a7-4c34-8f7f-fd023574aacc None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:49.334687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 530/1084] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:01:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:49.340745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a57bf1-39a7-4c34-8f7f-fd023574aacc None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:49.404290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd005f7-afd7-49a6-bc58-24bf853ec50a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:49.436188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a57bf1-39a7-4c34-8f7f-fd023574aacc None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:49.436654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1ab68-a41d-475e-b035-d5f89d4f1e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:49.437310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1ab68-a41d-475e-b035-d5f89d4f1e06 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.437597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1ab68-a41d-475e-b035-d5f89d4f1e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.437895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1ab68-a41d-475e-b035-d5f89d4f1e06 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.438641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1ab68-a41d-475e-b035-d5f89d4f1e06 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:49.438977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1ab68-a41d-475e-b035-d5f89d4f1e06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:49.440357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d1ab68-a41d-475e-b035-d5f89d4f1e06 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:49.448966 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 555/1085] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 5 15:01:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:49.454445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1ab68-a41d-475e-b035-d5f89d4f1e06 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:49.461758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:49.465241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:49.470978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78fab4-14fb-4058-970a-a3b9b83eb72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:49.471901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78fab4-14fb-4058-970a-a3b9b83eb72e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.472191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78fab4-14fb-4058-970a-a3b9b83eb72e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.472462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78fab4-14fb-4058-970a-a3b9b83eb72e None admin] PATH_INFO: `/v3/users/566278d9375f4af3ae6324f800ca7a0c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.473656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78fab4-14fb-4058-970a-a3b9b83eb72e None admin] RBAC: Authorizing `identity:get_user(user_id=566278d9375f4af3ae6324f800ca7a0c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:49.475115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78fab4-14fb-4058-970a-a3b9b83eb72e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:49.476726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 531/1086] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:01:49 2022] GET /identity/v3/users/566278d9375f4af3ae6324f800ca7a0c => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:49.482775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78fab4-14fb-4058-970a-a3b9b83eb72e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:49.553420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1ab68-a41d-475e-b035-d5f89d4f1e06 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:49.578600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78fab4-14fb-4058-970a-a3b9b83eb72e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:49.586803 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea8e3f-b24f-422a-a52a-676b58e83828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:49.587696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea8e3f-b24f-422a-a52a-676b58e83828 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.587987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea8e3f-b24f-422a-a52a-676b58e83828 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.588270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea8e3f-b24f-422a-a52a-676b58e83828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.598954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea8e3f-b24f-422a-a52a-676b58e83828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:49.600458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea8e3f-b24f-422a-a52a-676b58e83828 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:49.611306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190742a-8906-4846-9f62-a1347f6fc3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:49.612145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190742a-8906-4846-9f62-a1347f6fc3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.612429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190742a-8906-4846-9f62-a1347f6fc3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.612706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190742a-8906-4846-9f62-a1347f6fc3c6 None admin] PATH_INFO: `/v3/projects/57f118ac394a4d54b942f4e166623230` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.613854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190742a-8906-4846-9f62-a1347f6fc3c6 None admin] RBAC: Authorizing `identity:get_project(project_id=57f118ac394a4d54b942f4e166623230)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:49.615273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f190742a-8906-4846-9f62-a1347f6fc3c6 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:49.616928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 532/1087] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:01:49 2022] GET /identity/v3/projects/57f118ac394a4d54b942f4e166623230 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-b8ea8e3f-b24f-422a-a52a-676b58e83828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:49.617563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:49.619185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:49.620105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:49.620105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:49.620105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:49.620105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:49.620105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 556/1088] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:49.623262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190742a-8906-4846-9f62-a1347f6fc3c6 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:49.624821 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea8e3f-b24f-422a-a52a-676b58e83828 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:49.681641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:49.682598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:49.720914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190742a-8906-4846-9f62-a1347f6fc3c6 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:49.723873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea8e3f-b24f-422a-a52a-676b58e83828 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:49.753257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a49c01-c45c-4ee4-8b67-675244c4e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:49.754136 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a49c01-c45c-4ee4-8b67-675244c4e490 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.754421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a49c01-c45c-4ee4-8b67-675244c4e490 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.754701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a49c01-c45c-4ee4-8b67-675244c4e490 None admin] PATH_INFO: `/v3/projects/57f118ac394a4d54b942f4e166623230/users/566278d9375f4af3ae6324f800ca7a0c/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.756071 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b9cb7-e5c5-4548-a03a-878e21c9f1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:49.756707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a49c01-c45c-4ee4-8b67-675244c4e490 None admin] RBAC: Authorizing `identity:create_grant(project_id=57f118ac394a4d54b942f4e166623230, user_id=566278d9375f4af3ae6324f800ca7a0c, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:49.756813 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b9cb7-e5c5-4548-a03a-878e21c9f1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.756993 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b9cb7-e5c5-4548-a03a-878e21c9f1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.757271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b9cb7-e5c5-4548-a03a-878e21c9f1d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.757988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b9cb7-e5c5-4548-a03a-878e21c9f1d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:49.758134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a49c01-c45c-4ee4-8b67-675244c4e490 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:49.758297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b9cb7-e5c5-4548-a03a-878e21c9f1d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:49.760199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b9cb7-e5c5-4548-a03a-878e21c9f1d2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:49.772468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 557/1089] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:01:49.772749 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 533/1090] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:01:49 2022] PUT /identity/v3/projects/57f118ac394a4d54b942f4e166623230/users/566278d9375f4af3ae6324f800ca7a0c/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:01:49.778547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a49c01-c45c-4ee4-8b67-675244c4e490 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:49.885024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a49c01-c45c-4ee4-8b67-675244c4e490 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:49.909497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:49.927123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177bea1-c408-4f72-902d-84439e460316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:49.927925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177bea1-c408-4f72-902d-84439e460316 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:49.928208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177bea1-c408-4f72-902d-84439e460316 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:49.928484 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177bea1-c408-4f72-902d-84439e460316 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:49.937728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177bea1-c408-4f72-902d-84439e460316 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:49.939214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6177bea1-c408-4f72-902d-84439e460316 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-6177bea1-c408-4f72-902d-84439e460316 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:01:49.953847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:49.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:49.955815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 534/1091] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 5 15:01:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:01:49.960999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b9cb7-e5c5-4548-a03a-878e21c9f1d2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:50.057162 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b9cb7-e5c5-4548-a03a-878e21c9f1d2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:50.076619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:50.091087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93806df-ce67-4ca6-bb84-915cfa0711d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:50.091759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93806df-ce67-4ca6-bb84-915cfa0711d6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:50.092043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93806df-ce67-4ca6-bb84-915cfa0711d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:50.092323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93806df-ce67-4ca6-bb84-915cfa0711d6 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:50.093157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93806df-ce67-4ca6-bb84-915cfa0711d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:50.093468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93806df-ce67-4ca6-bb84-915cfa0711d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:50.094904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93806df-ce67-4ca6-bb84-915cfa0711d6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:50.111561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 558/1092] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 5 15:01:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:50.117272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177bea1-c408-4f72-902d-84439e460316 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:50.215230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177bea1-c408-4f72-902d-84439e460316 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:50.248115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c292911-8dcc-4dc2-8bf1-2fba5e658147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:50.248756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c292911-8dcc-4dc2-8bf1-2fba5e658147 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:50.249039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c292911-8dcc-4dc2-8bf1-2fba5e658147 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:50.249315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c292911-8dcc-4dc2-8bf1-2fba5e658147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:50.299582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:50.315564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c292911-8dcc-4dc2-8bf1-2fba5e658147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:50.316953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c292911-8dcc-4dc2-8bf1-2fba5e658147 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:50.412137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 535/1093] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:01:50.417814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93806df-ce67-4ca6-bb84-915cfa0711d6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:50.519401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93806df-ce67-4ca6-bb84-915cfa0711d6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:50.552545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa81b0-7fda-437f-937a-127bce080d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:50.553423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa81b0-7fda-437f-937a-127bce080d0b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:50.553775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa81b0-7fda-437f-937a-127bce080d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:50.554139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa81b0-7fda-437f-937a-127bce080d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:50.564716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa81b0-7fda-437f-937a-127bce080d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:50.566212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa81b0-7fda-437f-937a-127bce080d0b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-f0fa81b0-7fda-437f-937a-127bce080d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:01:50.583296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:50.584171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:50.585088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:01:50.585088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:01:50.585088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:01:50.585088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:50.585088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 559/1094] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:50.590071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c292911-8dcc-4dc2-8bf1-2fba5e658147 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:50.669595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:50.685411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c292911-8dcc-4dc2-8bf1-2fba5e658147 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:50.718527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de08ba7-8b50-49b6-8075-9efd13fa9c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:50.719212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de08ba7-8b50-49b6-8075-9efd13fa9c6f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:50.719499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de08ba7-8b50-49b6-8075-9efd13fa9c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:50.719776 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de08ba7-8b50-49b6-8075-9efd13fa9c6f None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:50.720496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de08ba7-8b50-49b6-8075-9efd13fa9c6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:50.720808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de08ba7-8b50-49b6-8075-9efd13fa9c6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:50.722169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de08ba7-8b50-49b6-8075-9efd13fa9c6f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:50.733865 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 536/1095] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:50.739187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa81b0-7fda-437f-937a-127bce080d0b None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:50.768188 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:50.837799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa81b0-7fda-437f-937a-127bce080d0b None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:50.873014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd8490-2e89-4e93-b259-8d8d8ee8f7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:50.873784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd8490-2e89-4e93-b259-8d8d8ee8f7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:50.874075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd8490-2e89-4e93-b259-8d8d8ee8f7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:50.874349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd8490-2e89-4e93-b259-8d8d8ee8f7e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:50.881075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd8490-2e89-4e93-b259-8d8d8ee8f7e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:50.882512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd8490-2e89-4e93-b259-8d8d8ee8f7e3 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-47cd8490-2e89-4e93-b259-8d8d8ee8f7e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:01:50.896048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:50.897094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:50.898402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:01:50.898402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:01:50.898402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:01:50.898402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:01:50.898402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:01:50.898402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:01:50.898402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:01:50.898402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:01:50.898402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 560/1096] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:01:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:50.903597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de08ba7-8b50-49b6-8075-9efd13fa9c6f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:50.999473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de08ba7-8b50-49b6-8075-9efd13fa9c6f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:51.031986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16353e56-ff1c-4df7-bb1f-87caa29f1b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:51.032633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16353e56-ff1c-4df7-bb1f-87caa29f1b76 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:51.032916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16353e56-ff1c-4df7-bb1f-87caa29f1b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:51.033190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16353e56-ff1c-4df7-bb1f-87caa29f1b76 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:51.033970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16353e56-ff1c-4df7-bb1f-87caa29f1b76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:51.034279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16353e56-ff1c-4df7-bb1f-87caa29f1b76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:51.035730 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16353e56-ff1c-4df7-bb1f-87caa29f1b76 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:51.047208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 537/1097] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:51.053085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd8490-2e89-4e93-b259-8d8d8ee8f7e3 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:51.140027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:51.150308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd8490-2e89-4e93-b259-8d8d8ee8f7e3 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:51.182493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51307511-ba41-4565-993d-2f8a43ac2f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:51.183191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51307511-ba41-4565-993d-2f8a43ac2f80 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:51.183476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51307511-ba41-4565-993d-2f8a43ac2f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:51.183753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51307511-ba41-4565-993d-2f8a43ac2f80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:51.184599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51307511-ba41-4565-993d-2f8a43ac2f80 None admin] RBAC: Adding query filter params (role.id=e08bb5ad422e4f6593bcbaca2bdf4366, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=e7a416f22d33450183801cef2b13822f) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:51.184914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51307511-ba41-4565-993d-2f8a43ac2f80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:51.186357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51307511-ba41-4565-993d-2f8a43ac2f80 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:51.203314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 561/1098] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:51 2022] GET /identity/v3/role_assignments?user.id=e7a416f22d33450183801cef2b13822f&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=e08bb5ad422e4f6593bcbaca2bdf4366 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:51.771940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a25fa-41a0-4a5c-9813-b4492b637821 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:51.772231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a25fa-41a0-4a5c-9813-b4492b637821 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:51.772693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a25fa-41a0-4a5c-9813-b4492b637821 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:51.774230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 538/1099] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:51.779496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394d005-875f-4bf3-8852-0161d706c4ea None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:51.779794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394d005-875f-4bf3-8852-0161d706c4ea None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:51.780080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394d005-875f-4bf3-8852-0161d706c4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:51.798247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-6394d005-875f-4bf3-8852-0161d706c4ea None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:51.799488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6394d005-875f-4bf3-8852-0161d706c4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:51.866541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 562/1100] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:51.890109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a25fa-41a0-4a5c-9813-b4492b637821 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:51.893975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:51.992290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a25fa-41a0-4a5c-9813-b4492b637821 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:52.026653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e4303-b5f0-471a-9032-152111f0d4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:52.027397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e4303-b5f0-471a-9032-152111f0d4da None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:52.027736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e4303-b5f0-471a-9032-152111f0d4da None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:52.028013 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91e4303-b5f0-471a-9032-152111f0d4da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:52.037700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91e4303-b5f0-471a-9032-152111f0d4da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c91e4303-b5f0-471a-9032-152111f0d4da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:01:52.045197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:01:52.046421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:01:52.052055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 539/1101] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:01:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:01:52.052937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394d005-875f-4bf3-8852-0161d706c4ea None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:52.132242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:52.148215 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394d005-875f-4bf3-8852-0161d706c4ea None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:52.179808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cf871-a6dc-4572-a07a-80408d159026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:52.180458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf871-a6dc-4572-a07a-80408d159026 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:52.181275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf871-a6dc-4572-a07a-80408d159026 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:52.181549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02cf871-a6dc-4572-a07a-80408d159026 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:52.182287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cf871-a6dc-4572-a07a-80408d159026 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:52.182590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cf871-a6dc-4572-a07a-80408d159026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:52.183971 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02cf871-a6dc-4572-a07a-80408d159026 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:52.191713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 563/1102] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:01:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:52.197565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e4303-b5f0-471a-9032-152111f0d4da None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:52.258141 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:52.296160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91e4303-b5f0-471a-9032-152111f0d4da None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:52.327516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c13ae-e832-4958-ba59-cc2577a863ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:52.328357 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c13ae-e832-4958-ba59-cc2577a863ae None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:52.328635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c13ae-e832-4958-ba59-cc2577a863ae None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:52.328908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c13ae-e832-4958-ba59-cc2577a863ae None admin] PATH_INFO: `/v3/users/566278d9375f4af3ae6324f800ca7a0c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:52.330066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c13ae-e832-4958-ba59-cc2577a863ae None admin] RBAC: Authorizing `identity:get_user(user_id=566278d9375f4af3ae6324f800ca7a0c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:52.331517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c13ae-e832-4958-ba59-cc2577a863ae None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:52.333099 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 540/1103] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:01:52 2022] GET /identity/v3/users/566278d9375f4af3ae6324f800ca7a0c => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:52.339338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cf871-a6dc-4572-a07a-80408d159026 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:52.434048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02cf871-a6dc-4572-a07a-80408d159026 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:52.466118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb2528-8247-4552-bd20-4778d4aa7105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:52.466751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb2528-8247-4552-bd20-4778d4aa7105 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:52.467034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb2528-8247-4552-bd20-4778d4aa7105 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:52.467345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb2528-8247-4552-bd20-4778d4aa7105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:52.482192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:52.533554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb2528-8247-4552-bd20-4778d4aa7105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:52.535008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb2528-8247-4552-bd20-4778d4aa7105 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:52.627324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 564/1104] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:01:52.633156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c13ae-e832-4958-ba59-cc2577a863ae None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:52.728140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c13ae-e832-4958-ba59-cc2577a863ae None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:52.754745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:52.760562 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587a3a0-db1c-41ed-961f-4fa47ccd052a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:52.761393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a3a0-db1c-41ed-961f-4fa47ccd052a None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:52.761673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a3a0-db1c-41ed-961f-4fa47ccd052a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:52.761945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587a3a0-db1c-41ed-961f-4fa47ccd052a None admin] PATH_INFO: `/v3/projects/57f118ac394a4d54b942f4e166623230` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:52.763123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587a3a0-db1c-41ed-961f-4fa47ccd052a None admin] RBAC: Authorizing `identity:get_project(project_id=57f118ac394a4d54b942f4e166623230)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:52.764549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1587a3a0-db1c-41ed-961f-4fa47ccd052a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:52.766173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 541/1105] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:01:52 2022] GET /identity/v3/projects/57f118ac394a4d54b942f4e166623230 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:52.771980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb2528-8247-4552-bd20-4778d4aa7105 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:52.815697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:52.866623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb2528-8247-4552-bd20-4778d4aa7105 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:52.900263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443def83-0f34-4184-a899-213de910f880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:52.900849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443def83-0f34-4184-a899-213de910f880 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:52.901139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443def83-0f34-4184-a899-213de910f880 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:52.901451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443def83-0f34-4184-a899-213de910f880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:52.902277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443def83-0f34-4184-a899-213de910f880 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=57f118ac394a4d54b942f4e166623230, user.id=566278d9375f4af3ae6324f800ca7a0c) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:52.902592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443def83-0f34-4184-a899-213de910f880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:52.904070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443def83-0f34-4184-a899-213de910f880 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:52.909727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 565/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:52 2022] GET /identity/v3/role_assignments?user.id=566278d9375f4af3ae6324f800ca7a0c&scope.project.id=57f118ac394a4d54b942f4e166623230&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:54.917213 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fec83-b246-4ddd-8be0-5a1e21bb1884 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:54.917759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fec83-b246-4ddd-8be0-5a1e21bb1884 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:54.917759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fec83-b246-4ddd-8be0-5a1e21bb1884 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:54.919216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 542/1107] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:54.924296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab3ea9-2223-4fd7-bf54-aa5fcc101091 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:54.924690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab3ea9-2223-4fd7-bf54-aa5fcc101091 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:54.924863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab3ea9-2223-4fd7-bf54-aa5fcc101091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:54.941152 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-fdab3ea9-2223-4fd7-bf54-aa5fcc101091 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:54.942334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-fdab3ea9-2223-4fd7-bf54-aa5fcc101091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:55.006429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 566/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:55.031406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fec83-b246-4ddd-8be0-5a1e21bb1884 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:55.128729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fec83-b246-4ddd-8be0-5a1e21bb1884 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:55.160979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5487f8d-4512-484f-9711-ef35fc4425f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:55.161755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5487f8d-4512-484f-9711-ef35fc4425f1 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:55.162043 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5487f8d-4512-484f-9711-ef35fc4425f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:55.162324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5487f8d-4512-484f-9711-ef35fc4425f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:55.163508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5487f8d-4512-484f-9711-ef35fc4425f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:55.164928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5487f8d-4512-484f-9711-ef35fc4425f1 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:55.167037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 543/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:01:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:55.172738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab3ea9-2223-4fd7-bf54-aa5fcc101091 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:55.212728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:55.267705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab3ea9-2223-4fd7-bf54-aa5fcc101091 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:55.300487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecdfae-e04a-4a70-9096-806251ccbd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:55.301145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecdfae-e04a-4a70-9096-806251ccbd67 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:55.301433 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecdfae-e04a-4a70-9096-806251ccbd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:55.301711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecdfae-e04a-4a70-9096-806251ccbd67 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:55.302424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecdfae-e04a-4a70-9096-806251ccbd67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:55.303902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecdfae-e04a-4a70-9096-806251ccbd67 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:55.323834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 567/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:01:55 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:01:57.346454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9788de-51c6-4e1b-b597-8afaf2f20b7f None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:57.346963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9788de-51c6-4e1b-b597-8afaf2f20b7f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:57.347186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9788de-51c6-4e1b-b597-8afaf2f20b7f None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:57.348403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 544/1111] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:01:57.353717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17233f-ba69-4666-a335-ccfae2b718b3 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:57.354127 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17233f-ba69-4666-a335-ccfae2b718b3 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:57.354376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17233f-ba69-4666-a335-ccfae2b718b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:57.371055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-2c17233f-ba69-4666-a335-ccfae2b718b3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:01:57.372233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2c17233f-ba69-4666-a335-ccfae2b718b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:01:57.435401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 568/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:01:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:01:57.459033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9788de-51c6-4e1b-b597-8afaf2f20b7f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:57.523754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:57.556418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9788de-51c6-4e1b-b597-8afaf2f20b7f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:57.589296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9f140-0a34-4898-bff0-4b1f18a38d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:57.590130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9f140-0a34-4898-bff0-4b1f18a38d44 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:57.590416 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9f140-0a34-4898-bff0-4b1f18a38d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:57.590693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9f140-0a34-4898-bff0-4b1f18a38d44 None admin] PATH_INFO: `/v3/roles/4301bced0259446cafb79118a53a015e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:57.592175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9f140-0a34-4898-bff0-4b1f18a38d44 None admin] RBAC: Authorizing `identity:get_role(role_id=4301bced0259446cafb79118a53a015e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:57.593538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9f140-0a34-4898-bff0-4b1f18a38d44 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:57.594867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 545/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:01:57 2022] GET /identity/v3/roles/4301bced0259446cafb79118a53a015e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:57.600422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17233f-ba69-4666-a335-ccfae2b718b3 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:57.626114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:57.696842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17233f-ba69-4666-a335-ccfae2b718b3 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:57.729248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2218f-9b7a-4708-a031-60957f0cf190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:57.730045 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2218f-9b7a-4708-a031-60957f0cf190 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:57.730328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2218f-9b7a-4708-a031-60957f0cf190 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:57.730601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2218f-9b7a-4708-a031-60957f0cf190 None admin] PATH_INFO: `/v3/users/976c8174bc3048c39a37f75c59afad35` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:57.743563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2218f-9b7a-4708-a031-60957f0cf190 None admin] RBAC: Authorizing `identity:get_user(user_id=976c8174bc3048c39a37f75c59afad35)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:57.744998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2218f-9b7a-4708-a031-60957f0cf190 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:57.746589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 569/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:01:57 2022] GET /identity/v3/users/976c8174bc3048c39a37f75c59afad35 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:57.752065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9f140-0a34-4898-bff0-4b1f18a38d44 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:57.846260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9f140-0a34-4898-bff0-4b1f18a38d44 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:57.879100 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d1cb6-e9ae-4539-8626-79fb964eb048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:57.879734 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d1cb6-e9ae-4539-8626-79fb964eb048 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:57.880026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d1cb6-e9ae-4539-8626-79fb964eb048 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:57.880314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d1cb6-e9ae-4539-8626-79fb964eb048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:57.883659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:57.944232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d1cb6-e9ae-4539-8626-79fb964eb048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:57.945670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d1cb6-e9ae-4539-8626-79fb964eb048 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:58.039720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 546/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:01:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:01:58.045843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2218f-9b7a-4708-a031-60957f0cf190 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:58.126161 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:58.141960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2218f-9b7a-4708-a031-60957f0cf190 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:58.173764 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cef11a-9933-4f74-8eff-ae0c1b0a383b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:58.174618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cef11a-9933-4f74-8eff-ae0c1b0a383b None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:58.174899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cef11a-9933-4f74-8eff-ae0c1b0a383b None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:58.175199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cef11a-9933-4f74-8eff-ae0c1b0a383b None admin] PATH_INFO: `/v3/projects/57f118ac394a4d54b942f4e166623230` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:58.176355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cef11a-9933-4f74-8eff-ae0c1b0a383b None admin] RBAC: Authorizing `identity:get_project(project_id=57f118ac394a4d54b942f4e166623230)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:58.177772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cef11a-9933-4f74-8eff-ae0c1b0a383b None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:58.179450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 570/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:01:58 2022] GET /identity/v3/projects/57f118ac394a4d54b942f4e166623230 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:01:58.185931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d1cb6-e9ae-4539-8626-79fb964eb048 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:01:58.219480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:01:58.281559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d1cb6-e9ae-4539-8626-79fb964eb048 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:01:58.314196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4fc72c-23bf-451e-b6f1-108b0914fa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:01:58.314790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fc72c-23bf-451e-b6f1-108b0914fa2e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:01:58.315105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fc72c-23bf-451e-b6f1-108b0914fa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:01:58.315395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4fc72c-23bf-451e-b6f1-108b0914fa2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:01:58.316224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fc72c-23bf-451e-b6f1-108b0914fa2e None admin] RBAC: Adding query filter params (role.id=4301bced0259446cafb79118a53a015e, scope.project.id=57f118ac394a4d54b942f4e166623230, user.id=976c8174bc3048c39a37f75c59afad35) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:01:58.316542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fc72c-23bf-451e-b6f1-108b0914fa2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:01:58.317962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4fc72c-23bf-451e-b6f1-108b0914fa2e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:01:58.323767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 547/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:01:58 2022] GET /identity/v3/role_assignments?user.id=976c8174bc3048c39a37f75c59afad35&scope.project.id=57f118ac394a4d54b942f4e166623230&role.id=4301bced0259446cafb79118a53a015e => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:00.290700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79abfd-7c7a-4581-961b-7c3b95a7262f None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:00.291312 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79abfd-7c7a-4581-961b-7c3b95a7262f None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:00.291312 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79abfd-7c7a-4581-961b-7c3b95a7262f None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:00.292775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 571/1118] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:00.298032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47ae04-9699-4f62-81cd-15836f218b77 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:00.298485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47ae04-9699-4f62-81cd-15836f218b77 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:00.298594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47ae04-9699-4f62-81cd-15836f218b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:00.314899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0a47ae04-9699-4f62-81cd-15836f218b77 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:00.316113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0a47ae04-9699-4f62-81cd-15836f218b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:00.379945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 548/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:00.402383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79abfd-7c7a-4581-961b-7c3b95a7262f None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:00.504009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79abfd-7c7a-4581-961b-7c3b95a7262f None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:00.536847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9a2cc-6623-44be-9a26-8950f8f15580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:00.537696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a2cc-6623-44be-9a26-8950f8f15580 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:00.537980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a2cc-6623-44be-9a26-8950f8f15580 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:00.538255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9a2cc-6623-44be-9a26-8950f8f15580 None admin] PATH_INFO: `/v3/roles/4301bced0259446cafb79118a53a015e` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:00.539864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9a2cc-6623-44be-9a26-8950f8f15580 None admin] RBAC: Authorizing `identity:get_role(role_id=4301bced0259446cafb79118a53a015e)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:00.541243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9a2cc-6623-44be-9a26-8950f8f15580 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:00.542599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 572/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:02:00 2022] GET /identity/v3/roles/4301bced0259446cafb79118a53a015e => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:00.548388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47ae04-9699-4f62-81cd-15836f218b77 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:00.578143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:00.645889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47ae04-9699-4f62-81cd-15836f218b77 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:00.677740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491594ff-4d75-4f6a-985f-9fa3f6cadbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:00.678600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491594ff-4d75-4f6a-985f-9fa3f6cadbd7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:00.678887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491594ff-4d75-4f6a-985f-9fa3f6cadbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:00.679196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491594ff-4d75-4f6a-985f-9fa3f6cadbd7 None admin] PATH_INFO: `/v3/users/976c8174bc3048c39a37f75c59afad35` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:00.680915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491594ff-4d75-4f6a-985f-9fa3f6cadbd7 None admin] RBAC: Authorizing `identity:get_user(user_id=976c8174bc3048c39a37f75c59afad35)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:00.682350 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491594ff-4d75-4f6a-985f-9fa3f6cadbd7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:00.683981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 549/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:02:00 2022] GET /identity/v3/users/976c8174bc3048c39a37f75c59afad35 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:00.689636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9a2cc-6623-44be-9a26-8950f8f15580 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:00.710863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:00.786907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9a2cc-6623-44be-9a26-8950f8f15580 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:00.818766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a11db-f08f-4495-949a-a4593a087cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:00.819666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a11db-f08f-4495-949a-a4593a087cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:00.819953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a11db-f08f-4495-949a-a4593a087cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:00.820233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a11db-f08f-4495-949a-a4593a087cd2 None admin] PATH_INFO: `/v3/projects/57f118ac394a4d54b942f4e166623230` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:00.821955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a11db-f08f-4495-949a-a4593a087cd2 None admin] RBAC: Authorizing `identity:get_project(project_id=57f118ac394a4d54b942f4e166623230)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:00.823423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a11db-f08f-4495-949a-a4593a087cd2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:00.825369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 573/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:02:00 2022] GET /identity/v3/projects/57f118ac394a4d54b942f4e166623230 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:00.831494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491594ff-4d75-4f6a-985f-9fa3f6cadbd7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:00.926244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:00.930361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491594ff-4d75-4f6a-985f-9fa3f6cadbd7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:00.963759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d13f73-63c2-4c7b-bfa5-dd1867dc55b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:00.964625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d13f73-63c2-4c7b-bfa5-dd1867dc55b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:00.964905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d13f73-63c2-4c7b-bfa5-dd1867dc55b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:00.965333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d13f73-63c2-4c7b-bfa5-dd1867dc55b4 None admin] PATH_INFO: `/v3/projects/57f118ac394a4d54b942f4e166623230/users/976c8174bc3048c39a37f75c59afad35/roles/4301bced0259446cafb79118a53a015e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:00.967512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d13f73-63c2-4c7b-bfa5-dd1867dc55b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=57f118ac394a4d54b942f4e166623230, user_id=976c8174bc3048c39a37f75c59afad35, role_id=4301bced0259446cafb79118a53a015e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:00.968929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d13f73-63c2-4c7b-bfa5-dd1867dc55b4 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:00.980838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 550/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:02:00 2022] PUT /identity/v3/projects/57f118ac394a4d54b942f4e166623230/users/976c8174bc3048c39a37f75c59afad35/roles/4301bced0259446cafb79118a53a015e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:02:02.995003 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c4964-bc77-4071-b55f-1589ade22bdb None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:02.995586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c4964-bc77-4071-b55f-1589ade22bdb None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:02.995586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c4964-bc77-4071-b55f-1589ade22bdb None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:02.996957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 574/1124] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:03.002138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2326f-8e20-4590-98db-22e12c067e5e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:03.002544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2326f-8e20-4590-98db-22e12c067e5e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:03.002706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2326f-8e20-4590-98db-22e12c067e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:03.019542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-beb2326f-8e20-4590-98db-22e12c067e5e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:03.020728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-beb2326f-8e20-4590-98db-22e12c067e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:03.084414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 551/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:03.107958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c4964-bc77-4071-b55f-1589ade22bdb None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:03.191793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:03.205721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c4964-bc77-4071-b55f-1589ade22bdb None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:03.238634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856d3c3-b855-4060-abe4-136db2cacac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:03.239512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856d3c3-b855-4060-abe4-136db2cacac7 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:03.239802 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856d3c3-b855-4060-abe4-136db2cacac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:03.240081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856d3c3-b855-4060-abe4-136db2cacac7 None admin] PATH_INFO: `/v3/roles/4301bced0259446cafb79118a53a015e` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:03.241552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856d3c3-b855-4060-abe4-136db2cacac7 None admin] RBAC: Authorizing `identity:get_role(role_id=4301bced0259446cafb79118a53a015e)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:03.242918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856d3c3-b855-4060-abe4-136db2cacac7 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:03.244303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 575/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:02:03 2022] GET /identity/v3/roles/4301bced0259446cafb79118a53a015e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:03.249998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2326f-8e20-4590-98db-22e12c067e5e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:03.333579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:03.346496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2326f-8e20-4590-98db-22e12c067e5e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:03.379537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0543fd9-f7e1-48bd-a12f-80cf50efa02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:03.380385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0543fd9-f7e1-48bd-a12f-80cf50efa02b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:03.380671 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0543fd9-f7e1-48bd-a12f-80cf50efa02b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:03.380948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0543fd9-f7e1-48bd-a12f-80cf50efa02b None admin] PATH_INFO: `/v3/users/976c8174bc3048c39a37f75c59afad35` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:03.382118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0543fd9-f7e1-48bd-a12f-80cf50efa02b None admin] RBAC: Authorizing `identity:get_user(user_id=976c8174bc3048c39a37f75c59afad35)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:03.384166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0543fd9-f7e1-48bd-a12f-80cf50efa02b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:03.385743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 552/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:02:03 2022] GET /identity/v3/users/976c8174bc3048c39a37f75c59afad35 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:03.391264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856d3c3-b855-4060-abe4-136db2cacac7 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:03.485561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856d3c3-b855-4060-abe4-136db2cacac7 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:03.518367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f958ef7-1123-4aa9-9082-46d7a4563486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:03.518939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f958ef7-1123-4aa9-9082-46d7a4563486 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:03.519255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f958ef7-1123-4aa9-9082-46d7a4563486 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:03.519537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f958ef7-1123-4aa9-9082-46d7a4563486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:03.542844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:03.584788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f958ef7-1123-4aa9-9082-46d7a4563486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:03.586218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f958ef7-1123-4aa9-9082-46d7a4563486 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:03.678456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 576/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:02:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:02:03.684606 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0543fd9-f7e1-48bd-a12f-80cf50efa02b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:03.779584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0543fd9-f7e1-48bd-a12f-80cf50efa02b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:03.811977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a5c22-5773-456b-9422-f1fa47bc2e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:03.812810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5c22-5773-456b-9422-f1fa47bc2e9b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:03.813089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5c22-5773-456b-9422-f1fa47bc2e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:03.813362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a5c22-5773-456b-9422-f1fa47bc2e9b None admin] PATH_INFO: `/v3/projects/57f118ac394a4d54b942f4e166623230` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:03.814509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a5c22-5773-456b-9422-f1fa47bc2e9b None admin] RBAC: Authorizing `identity:get_project(project_id=57f118ac394a4d54b942f4e166623230)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:03.816018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a5c22-5773-456b-9422-f1fa47bc2e9b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:03.817655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 553/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:02:03 2022] GET /identity/v3/projects/57f118ac394a4d54b942f4e166623230 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:03.823606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f958ef7-1123-4aa9-9082-46d7a4563486 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:03.877902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:03.919272 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f958ef7-1123-4aa9-9082-46d7a4563486 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:03.952069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c3422-c4af-4c65-9da5-63e5c5c29e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:03.952713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c3422-c4af-4c65-9da5-63e5c5c29e5d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:03.953000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c3422-c4af-4c65-9da5-63e5c5c29e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:03.953280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c3422-c4af-4c65-9da5-63e5c5c29e5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:03.954109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c3422-c4af-4c65-9da5-63e5c5c29e5d None admin] RBAC: Adding query filter params (role.id=4301bced0259446cafb79118a53a015e, scope.project.id=57f118ac394a4d54b942f4e166623230, user.id=976c8174bc3048c39a37f75c59afad35) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:03.954427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c3422-c4af-4c65-9da5-63e5c5c29e5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:03.955888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c3422-c4af-4c65-9da5-63e5c5c29e5d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:03.961513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 577/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:02:03 2022] GET /identity/v3/role_assignments?user.id=976c8174bc3048c39a37f75c59afad35&scope.project.id=57f118ac394a4d54b942f4e166623230&role.id=4301bced0259446cafb79118a53a015e => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:05.967947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9f152-8f08-4dc6-bee0-1e8e10b2dc1a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:05.968497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9f152-8f08-4dc6-bee0-1e8e10b2dc1a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:05.968497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9f152-8f08-4dc6-bee0-1e8e10b2dc1a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:05.969847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 554/1131] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:05.974981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c67693-abd2-406f-a39c-bd864ecc0323 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:05.975437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c67693-abd2-406f-a39c-bd864ecc0323 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:05.975582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c67693-abd2-406f-a39c-bd864ecc0323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:05.992281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-17c67693-abd2-406f-a39c-bd864ecc0323 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:05.994075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-17c67693-abd2-406f-a39c-bd864ecc0323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:06.057338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 578/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:06.081385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9f152-8f08-4dc6-bee0-1e8e10b2dc1a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:06.105755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:06.178731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9f152-8f08-4dc6-bee0-1e8e10b2dc1a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:06.213349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd44209-2e58-42c5-9ef1-aae7d72caa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:06.214132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd44209-2e58-42c5-9ef1-aae7d72caa84 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:06.214417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd44209-2e58-42c5-9ef1-aae7d72caa84 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:06.214691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd44209-2e58-42c5-9ef1-aae7d72caa84 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:06.215871 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd44209-2e58-42c5-9ef1-aae7d72caa84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:06.217300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd44209-2e58-42c5-9ef1-aae7d72caa84 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:06.218954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 555/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:02:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:06.224632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c67693-abd2-406f-a39c-bd864ecc0323 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:06.271570 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:06.318821 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c67693-abd2-406f-a39c-bd864ecc0323 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:06.351558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc0a62-4228-4a45-b11c-82c43a251653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:06.352225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc0a62-4228-4a45-b11c-82c43a251653 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:06.352508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc0a62-4228-4a45-b11c-82c43a251653 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:06.352782 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc0a62-4228-4a45-b11c-82c43a251653 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:06.353491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc0a62-4228-4a45-b11c-82c43a251653 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:06.354906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc0a62-4228-4a45-b11c-82c43a251653 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:06.379258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 579/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:02:06 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:02:08.412614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a98bf-3da9-44c1-ac88-501e2c601bbf None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:08.413131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a98bf-3da9-44c1-ac88-501e2c601bbf None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:08.413242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032a98bf-3da9-44c1-ac88-501e2c601bbf None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:08.415057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 556/1135] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:08.420079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a9fb6-c2b8-4973-ab08-bab86c162af2 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:08.420482 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a9fb6-c2b8-4973-ab08-bab86c162af2 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:08.420649 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a9fb6-c2b8-4973-ab08-bab86c162af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:08.437555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d46a9fb6-c2b8-4973-ab08-bab86c162af2 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:08.438736 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d46a9fb6-c2b8-4973-ab08-bab86c162af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:08.502588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 580/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:08.526572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a98bf-3da9-44c1-ac88-501e2c601bbf None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:08.615843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:08.626271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032a98bf-3da9-44c1-ac88-501e2c601bbf None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:08.658967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366f2c7-4ce9-4470-b695-3b596c21fca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:08.659770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366f2c7-4ce9-4470-b695-3b596c21fca8 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:08.660059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366f2c7-4ce9-4470-b695-3b596c21fca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:08.660338 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366f2c7-4ce9-4470-b695-3b596c21fca8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:08.661508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366f2c7-4ce9-4470-b695-3b596c21fca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:08.662945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366f2c7-4ce9-4470-b695-3b596c21fca8 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:08.664646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 557/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:02:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:08.670278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a9fb6-c2b8-4973-ab08-bab86c162af2 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:08.700650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:08.766171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46a9fb6-c2b8-4973-ab08-bab86c162af2 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:08.798353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a1ce6-8ba2-4fd6-bca5-356be32e06e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:08.799008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a1ce6-8ba2-4fd6-bca5-356be32e06e7 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:08.799323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a1ce6-8ba2-4fd6-bca5-356be32e06e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:08.799601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a1ce6-8ba2-4fd6-bca5-356be32e06e7 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:08.800300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a1ce6-8ba2-4fd6-bca5-356be32e06e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:08.801726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a1ce6-8ba2-4fd6-bca5-356be32e06e7 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:08.820494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 581/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:02:08 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:02:10.844902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f63c44-bace-49cb-90eb-fed3ccee93e3 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:10.845430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f63c44-bace-49cb-90eb-fed3ccee93e3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:10.845430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f63c44-bace-49cb-90eb-fed3ccee93e3 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:10.846794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 558/1139] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:10.852325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792b4d4-5dd9-47f6-82aa-14517b5ac6f7 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:10.852716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792b4d4-5dd9-47f6-82aa-14517b5ac6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:10.852891 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792b4d4-5dd9-47f6-82aa-14517b5ac6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:10.869708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-2792b4d4-5dd9-47f6-82aa-14517b5ac6f7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:10.870885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2792b4d4-5dd9-47f6-82aa-14517b5ac6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:10.937522 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 582/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:10.960806 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f63c44-bace-49cb-90eb-fed3ccee93e3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:11.056968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f63c44-bace-49cb-90eb-fed3ccee93e3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:11.089489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a364552-595b-4375-8106-2c18642c8513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:11.090269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a364552-595b-4375-8106-2c18642c8513 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:11.090558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a364552-595b-4375-8106-2c18642c8513 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:11.090840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a364552-595b-4375-8106-2c18642c8513 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:11.100702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a364552-595b-4375-8106-2c18642c8513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-5a364552-595b-4375-8106-2c18642c8513 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:02:11.108841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:02:11.110014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:02:11.111182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 559/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:02:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:02:11.115801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792b4d4-5dd9-47f6-82aa-14517b5ac6f7 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:11.133546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:11.212229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792b4d4-5dd9-47f6-82aa-14517b5ac6f7 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:11.245147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ccbe8-12a4-4511-9dda-b212907901d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:11.245808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ccbe8-12a4-4511-9dda-b212907901d3 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:11.246098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ccbe8-12a4-4511-9dda-b212907901d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:11.246379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ccbe8-12a4-4511-9dda-b212907901d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:11.247163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ccbe8-12a4-4511-9dda-b212907901d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:11.247477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ccbe8-12a4-4511-9dda-b212907901d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:11.248852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ccbe8-12a4-4511-9dda-b212907901d3 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:11.256695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 583/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:02:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:11.262799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a364552-595b-4375-8106-2c18642c8513 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:11.284306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:11.360516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a364552-595b-4375-8106-2c18642c8513 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:11.392435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e57f23-38fb-4d3e-8f0c-75623197474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:11.393290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e57f23-38fb-4d3e-8f0c-75623197474c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:11.393576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e57f23-38fb-4d3e-8f0c-75623197474c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:11.393854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e57f23-38fb-4d3e-8f0c-75623197474c None admin] PATH_INFO: `/v3/users/f797f22cbb724b35bc82e680b4af4569` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:11.406481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e57f23-38fb-4d3e-8f0c-75623197474c None admin] RBAC: Authorizing `identity:get_user(user_id=f797f22cbb724b35bc82e680b4af4569)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:11.407947 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e57f23-38fb-4d3e-8f0c-75623197474c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:11.409551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 560/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:02:11 2022] GET /identity/v3/users/f797f22cbb724b35bc82e680b4af4569 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:11.415496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ccbe8-12a4-4511-9dda-b212907901d3 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:11.495000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:11.513529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ccbe8-12a4-4511-9dda-b212907901d3 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:11.545949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2218-d5eb-4278-941c-1b856b28e840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:11.546588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2218-d5eb-4278-941c-1b856b28e840 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:11.546874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2218-d5eb-4278-941c-1b856b28e840 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:11.547182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd2218-d5eb-4278-941c-1b856b28e840 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:11.611752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd2218-d5eb-4278-941c-1b856b28e840 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:11.613138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd2218-d5eb-4278-941c-1b856b28e840 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:11.687828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:11.706936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 584/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:02:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:02:11.713127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e57f23-38fb-4d3e-8f0c-75623197474c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:11.788630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:11.809690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e57f23-38fb-4d3e-8f0c-75623197474c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:11.842378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589059e-32c2-43d7-ad8e-28445e86d783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:11.843247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589059e-32c2-43d7-ad8e-28445e86d783 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:11.843532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589059e-32c2-43d7-ad8e-28445e86d783 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:11.843810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589059e-32c2-43d7-ad8e-28445e86d783 None admin] PATH_INFO: `/v3/projects/4af600963dfe47c49d4aa7414b76c187` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:11.844975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589059e-32c2-43d7-ad8e-28445e86d783 None admin] RBAC: Authorizing `identity:get_project(project_id=4af600963dfe47c49d4aa7414b76c187)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:11.846403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589059e-32c2-43d7-ad8e-28445e86d783 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:11.848087 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 561/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:02:11 2022] GET /identity/v3/projects/4af600963dfe47c49d4aa7414b76c187 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:11.854432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2218-d5eb-4278-941c-1b856b28e840 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:11.947286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd2218-d5eb-4278-941c-1b856b28e840 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:11.980450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe31674-299b-4277-869b-0acf1f5cff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:11.981091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe31674-299b-4277-869b-0acf1f5cff81 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:11.981378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe31674-299b-4277-869b-0acf1f5cff81 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:11.981656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe31674-299b-4277-869b-0acf1f5cff81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:11.982484 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe31674-299b-4277-869b-0acf1f5cff81 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=4af600963dfe47c49d4aa7414b76c187, user.id=f797f22cbb724b35bc82e680b4af4569) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:11.982799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe31674-299b-4277-869b-0acf1f5cff81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:11.984260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe31674-299b-4277-869b-0acf1f5cff81 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:11.989540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 585/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:02:11 2022] GET /identity/v3/role_assignments?user.id=f797f22cbb724b35bc82e680b4af4569&scope.project.id=4af600963dfe47c49d4aa7414b76c187&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:13.958730 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078ece8-f8de-4311-bbc9-c486b0a147f7 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:13.959322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078ece8-f8de-4311-bbc9-c486b0a147f7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:13.959322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078ece8-f8de-4311-bbc9-c486b0a147f7 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:13.960689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 562/1147] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:13.966298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6de3f1-11f2-4fde-8af3-483044a3ee73 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:13.966711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6de3f1-11f2-4fde-8af3-483044a3ee73 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:13.966854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6de3f1-11f2-4fde-8af3-483044a3ee73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:13.983847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4e6de3f1-11f2-4fde-8af3-483044a3ee73 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:13.985014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4e6de3f1-11f2-4fde-8af3-483044a3ee73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:14.022021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:14.049155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 586/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:14.070926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078ece8-f8de-4311-bbc9-c486b0a147f7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:14.168778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078ece8-f8de-4311-bbc9-c486b0a147f7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:14.200950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5110d-a141-4f92-b3da-729dbd3610aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:14.201735 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5110d-a141-4f92-b3da-729dbd3610aa None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:14.202015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5110d-a141-4f92-b3da-729dbd3610aa None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:14.202288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d5110d-a141-4f92-b3da-729dbd3610aa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:14.212675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d5110d-a141-4f92-b3da-729dbd3610aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-02d5110d-a141-4f92-b3da-729dbd3610aa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:02:14.220193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:02:14.221290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:02:14.222334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 563/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:02:14.228115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6de3f1-11f2-4fde-8af3-483044a3ee73 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:14.322048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6de3f1-11f2-4fde-8af3-483044a3ee73 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:14.354583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25d498-69a0-43ce-a52a-333c72a8544a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:14.355272 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25d498-69a0-43ce-a52a-333c72a8544a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:14.355558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25d498-69a0-43ce-a52a-333c72a8544a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:14.355833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d25d498-69a0-43ce-a52a-333c72a8544a None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:14.356569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25d498-69a0-43ce-a52a-333c72a8544a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:14.356876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25d498-69a0-43ce-a52a-333c72a8544a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:14.358242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d25d498-69a0-43ce-a52a-333c72a8544a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:14.365914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 587/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:14.371661 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5110d-a141-4f92-b3da-729dbd3610aa None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:14.380850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:14.468760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d5110d-a141-4f92-b3da-729dbd3610aa None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:14.499988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f82fae-a8c4-4823-88a7-630825616823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:14.500836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f82fae-a8c4-4823-88a7-630825616823 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:14.501122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f82fae-a8c4-4823-88a7-630825616823 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:14.501401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f82fae-a8c4-4823-88a7-630825616823 None admin] PATH_INFO: `/v3/users/f797f22cbb724b35bc82e680b4af4569` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:14.502570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f82fae-a8c4-4823-88a7-630825616823 None admin] RBAC: Authorizing `identity:get_user(user_id=f797f22cbb724b35bc82e680b4af4569)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:14.504015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f82fae-a8c4-4823-88a7-630825616823 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:14.505587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 564/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:02:14 2022] GET /identity/v3/users/f797f22cbb724b35bc82e680b4af4569 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:14.511754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25d498-69a0-43ce-a52a-333c72a8544a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:14.545774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:14.608242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d25d498-69a0-43ce-a52a-333c72a8544a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:14.640502 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe38975-94f4-4c9d-99c3-a11353313e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:14.641350 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe38975-94f4-4c9d-99c3-a11353313e6f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:14.641632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe38975-94f4-4c9d-99c3-a11353313e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:14.641905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe38975-94f4-4c9d-99c3-a11353313e6f None admin] PATH_INFO: `/v3/projects/4af600963dfe47c49d4aa7414b76c187` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:14.643068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe38975-94f4-4c9d-99c3-a11353313e6f None admin] RBAC: Authorizing `identity:get_project(project_id=4af600963dfe47c49d4aa7414b76c187)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:14.644507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe38975-94f4-4c9d-99c3-a11353313e6f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:14.646208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 588/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:02:14 2022] GET /identity/v3/projects/4af600963dfe47c49d4aa7414b76c187 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:14.652134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f82fae-a8c4-4823-88a7-630825616823 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:14.724476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:14.748557 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f82fae-a8c4-4823-88a7-630825616823 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:14.780432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8199e-fc3b-4e87-b767-531f7fa9287f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:14.781309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8199e-fc3b-4e87-b767-531f7fa9287f None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:14.781596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8199e-fc3b-4e87-b767-531f7fa9287f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:14.781877 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8199e-fc3b-4e87-b767-531f7fa9287f None admin] PATH_INFO: `/v3/projects/4af600963dfe47c49d4aa7414b76c187/users/f797f22cbb724b35bc82e680b4af4569/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:14.783836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8199e-fc3b-4e87-b767-531f7fa9287f None admin] RBAC: Authorizing `identity:create_grant(project_id=4af600963dfe47c49d4aa7414b76c187, user_id=f797f22cbb724b35bc82e680b4af4569, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:14.785257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8199e-fc3b-4e87-b767-531f7fa9287f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:14.797767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 565/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:02:14 2022] PUT /identity/v3/projects/4af600963dfe47c49d4aa7414b76c187/users/f797f22cbb724b35bc82e680b4af4569/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:02:16.788416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb94a0e-3921-4ded-90e2-8fd2d3d2183b None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:16.789471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb94a0e-3921-4ded-90e2-8fd2d3d2183b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:16.789471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb94a0e-3921-4ded-90e2-8fd2d3d2183b None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:16.790821 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 589/1154] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:16.796053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd8577-2e39-4351-b270-ff0437b166a4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:16.796439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd8577-2e39-4351-b270-ff0437b166a4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:16.796620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd8577-2e39-4351-b270-ff0437b166a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:16.813955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5dbd8577-2e39-4351-b270-ff0437b166a4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:16.815191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5dbd8577-2e39-4351-b270-ff0437b166a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:16.879329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 566/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:16.902603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb94a0e-3921-4ded-90e2-8fd2d3d2183b None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:17.001358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb94a0e-3921-4ded-90e2-8fd2d3d2183b None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:17.003633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:17.033697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da662645-39cc-4a92-8e1d-6c61cad3f1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:17.034470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da662645-39cc-4a92-8e1d-6c61cad3f1fd None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:17.034752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da662645-39cc-4a92-8e1d-6c61cad3f1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:17.035029 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da662645-39cc-4a92-8e1d-6c61cad3f1fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:17.045724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da662645-39cc-4a92-8e1d-6c61cad3f1fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-da662645-39cc-4a92-8e1d-6c61cad3f1fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:02:17.053259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:02:17.054373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:02:17.055616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 590/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:02:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:02:17.060231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd8577-2e39-4351-b270-ff0437b166a4 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:17.119269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:17.157294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd8577-2e39-4351-b270-ff0437b166a4 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:17.189873 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebce293-c79f-49f4-b906-b04d3fe3c3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:17.190561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebce293-c79f-49f4-b906-b04d3fe3c3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:17.190833 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebce293-c79f-49f4-b906-b04d3fe3c3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:17.191147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebce293-c79f-49f4-b906-b04d3fe3c3c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:17.191896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebce293-c79f-49f4-b906-b04d3fe3c3c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:17.192208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebce293-c79f-49f4-b906-b04d3fe3c3c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:17.193594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebce293-c79f-49f4-b906-b04d3fe3c3c2 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:17.201993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 567/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:02:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:17.207620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da662645-39cc-4a92-8e1d-6c61cad3f1fd None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:17.302995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da662645-39cc-4a92-8e1d-6c61cad3f1fd None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:17.335415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af875096-f581-451f-ac17-8eed27268811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:17.336333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af875096-f581-451f-ac17-8eed27268811 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:17.336616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af875096-f581-451f-ac17-8eed27268811 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:17.336890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af875096-f581-451f-ac17-8eed27268811 None admin] PATH_INFO: `/v3/users/f797f22cbb724b35bc82e680b4af4569` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:17.338046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af875096-f581-451f-ac17-8eed27268811 None admin] RBAC: Authorizing `identity:get_user(user_id=f797f22cbb724b35bc82e680b4af4569)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:17.339505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af875096-f581-451f-ac17-8eed27268811 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:17.341079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 591/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:02:17 2022] GET /identity/v3/users/f797f22cbb724b35bc82e680b4af4569 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:17.346673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebce293-c79f-49f4-b906-b04d3fe3c3c2 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:17.443337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebce293-c79f-49f4-b906-b04d3fe3c3c2 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:17.467242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:17.475712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee23b7-aa55-401e-83a6-6c3377031d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:17.476351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee23b7-aa55-401e-83a6-6c3377031d2f None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:17.477147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee23b7-aa55-401e-83a6-6c3377031d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:17.477430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee23b7-aa55-401e-83a6-6c3377031d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:17.541500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee23b7-aa55-401e-83a6-6c3377031d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:17.542942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ee23b7-aa55-401e-83a6-6c3377031d2f None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:17.636681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 568/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:02:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:02:17.642267 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af875096-f581-451f-ac17-8eed27268811 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:17.672534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:17.737627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af875096-f581-451f-ac17-8eed27268811 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:17.770644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb04c80-d6cb-4494-888f-3cdf14bb0d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:17.771521 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb04c80-d6cb-4494-888f-3cdf14bb0d4d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:17.771810 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb04c80-d6cb-4494-888f-3cdf14bb0d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:17.772091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb04c80-d6cb-4494-888f-3cdf14bb0d4d None admin] PATH_INFO: `/v3/projects/4af600963dfe47c49d4aa7414b76c187` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:17.773259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb04c80-d6cb-4494-888f-3cdf14bb0d4d None admin] RBAC: Authorizing `identity:get_project(project_id=4af600963dfe47c49d4aa7414b76c187)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:17.774685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb04c80-d6cb-4494-888f-3cdf14bb0d4d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:17.776390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 592/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:02:17 2022] GET /identity/v3/projects/4af600963dfe47c49d4aa7414b76c187 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:17.782192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee23b7-aa55-401e-83a6-6c3377031d2f None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:17.808091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:17.878252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee23b7-aa55-401e-83a6-6c3377031d2f None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:17.910571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed806cf2-6003-481d-b6c7-3c7a7e2b0130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:17.911180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed806cf2-6003-481d-b6c7-3c7a7e2b0130 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:17.911468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed806cf2-6003-481d-b6c7-3c7a7e2b0130 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:17.911749 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed806cf2-6003-481d-b6c7-3c7a7e2b0130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:17.912639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed806cf2-6003-481d-b6c7-3c7a7e2b0130 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=4af600963dfe47c49d4aa7414b76c187, user.id=f797f22cbb724b35bc82e680b4af4569) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:17.912963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed806cf2-6003-481d-b6c7-3c7a7e2b0130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:17.914411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed806cf2-6003-481d-b6c7-3c7a7e2b0130 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:17.919999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 569/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:02:17 2022] GET /identity/v3/role_assignments?user.id=f797f22cbb724b35bc82e680b4af4569&scope.project.id=4af600963dfe47c49d4aa7414b76c187&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:19.908959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbffe0a-6c95-4549-aca5-b929fc020168 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:19.909559 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbffe0a-6c95-4549-aca5-b929fc020168 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:19.909559 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbffe0a-6c95-4549-aca5-b929fc020168 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:19.910875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 593/1162] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:19.916053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbda2a7-9f30-4ff0-acc2-f27f22466784 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:19.916487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbda2a7-9f30-4ff0-acc2-f27f22466784 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:19.916621 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbda2a7-9f30-4ff0-acc2-f27f22466784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:19.933564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4cbda2a7-9f30-4ff0-acc2-f27f22466784 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:19.934763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4cbda2a7-9f30-4ff0-acc2-f27f22466784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:19.998468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 570/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:20.020139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbffe0a-6c95-4549-aca5-b929fc020168 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:20.113203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:20.117942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbffe0a-6c95-4549-aca5-b929fc020168 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:20.150292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6067bba-d162-44ad-a274-c0d1acb79f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:20.150931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6067bba-d162-44ad-a274-c0d1acb79f93 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:20.151248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6067bba-d162-44ad-a274-c0d1acb79f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:20.151524 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6067bba-d162-44ad-a274-c0d1acb79f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:20.222009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6067bba-d162-44ad-a274-c0d1acb79f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:20.223515 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6067bba-d162-44ad-a274-c0d1acb79f93 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:20.315314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:20.319385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 594/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:02:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:02:20.325402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbda2a7-9f30-4ff0-acc2-f27f22466784 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:20.345500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:20.419652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbda2a7-9f30-4ff0-acc2-f27f22466784 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:20.451983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850785ff-35af-41f6-9914-a7e66185e7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:20.452709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850785ff-35af-41f6-9914-a7e66185e7b3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:20.452992 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850785ff-35af-41f6-9914-a7e66185e7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:20.453266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850785ff-35af-41f6-9914-a7e66185e7b3 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:20.459670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850785ff-35af-41f6-9914-a7e66185e7b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:20.461131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850785ff-35af-41f6-9914-a7e66185e7b3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-850785ff-35af-41f6-9914-a7e66185e7b3 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:02:20.474733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:02:20.475839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:02:20.477337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 571/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 5 15:02:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:02:20.482308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6067bba-d162-44ad-a274-c0d1acb79f93 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:20.576956 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6067bba-d162-44ad-a274-c0d1acb79f93 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:20.609299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915f777-7e00-4175-9f6c-ecd4a11e9746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:20.609938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915f777-7e00-4175-9f6c-ecd4a11e9746 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:20.610221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915f777-7e00-4175-9f6c-ecd4a11e9746 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:20.610494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915f777-7e00-4175-9f6c-ecd4a11e9746 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:20.611233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915f777-7e00-4175-9f6c-ecd4a11e9746 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:20.611539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915f777-7e00-4175-9f6c-ecd4a11e9746 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:20.612907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915f777-7e00-4175-9f6c-ecd4a11e9746 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:20.620491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 595/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 5 15:02:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:22.588926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1a009-bbce-480f-95fc-a99794ff074c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:22.589427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1a009-bbce-480f-95fc-a99794ff074c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:22.589427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1a009-bbce-480f-95fc-a99794ff074c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:22.590785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 572/1167] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:22.602690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719ed04-83f8-4412-988c-9087aac0e807 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:22.603139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719ed04-83f8-4412-988c-9087aac0e807 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:22.603278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719ed04-83f8-4412-988c-9087aac0e807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:22.620181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-9719ed04-83f8-4412-988c-9087aac0e807 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:22.621364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9719ed04-83f8-4412-988c-9087aac0e807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:22.677180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:22.686303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 596/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:22.709398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1a009-bbce-480f-95fc-a99794ff074c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:22.789809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:22.809579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1a009-bbce-480f-95fc-a99794ff074c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:22.843035 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602a1c7-5c99-461f-8a37-fe537c134644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:22.843656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a1c7-5c99-461f-8a37-fe537c134644 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:22.843943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a1c7-5c99-461f-8a37-fe537c134644 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:22.844220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a1c7-5c99-461f-8a37-fe537c134644 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:22.844950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602a1c7-5c99-461f-8a37-fe537c134644 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:22.846319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a602a1c7-5c99-461f-8a37-fe537c134644 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:22.862278 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 573/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Tue Jul 5 15:02:22 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:02:24.878366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017ab45-fa39-442e-ba3e-270c4c079212 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:24.878918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017ab45-fa39-442e-ba3e-270c4c079212 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:24.878918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017ab45-fa39-442e-ba3e-270c4c079212 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:24.880362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 597/1170] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:24.886128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69ffb7-0ff8-4246-b0bc-4d6e992ea393 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:24.886501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69ffb7-0ff8-4246-b0bc-4d6e992ea393 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:24.886683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69ffb7-0ff8-4246-b0bc-4d6e992ea393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:24.903435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3a69ffb7-0ff8-4246-b0bc-4d6e992ea393 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:24.904604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3a69ffb7-0ff8-4246-b0bc-4d6e992ea393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:24.968061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 574/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:24.990796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017ab45-fa39-442e-ba3e-270c4c079212 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:25.088642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017ab45-fa39-442e-ba3e-270c4c079212 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:25.120748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8ef7f-26ad-4425-bffc-07c23939e06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:25.121632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8ef7f-26ad-4425-bffc-07c23939e06d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:25.121914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8ef7f-26ad-4425-bffc-07c23939e06d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:25.122190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8ef7f-26ad-4425-bffc-07c23939e06d None admin] PATH_INFO: `/v3/domains/3430703476c34e1ea3010f60a470b260` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:25.132237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8ef7f-26ad-4425-bffc-07c23939e06d None admin] RBAC: Authorizing `identity:get_domain(domain_id=3430703476c34e1ea3010f60a470b260)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:25.134257 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8ef7f-26ad-4425-bffc-07c23939e06d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:25.135880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 598/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Tue Jul 5 15:02:24 2022] GET /identity/v3/domains/3430703476c34e1ea3010f60a470b260 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:25.141693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69ffb7-0ff8-4246-b0bc-4d6e992ea393 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:25.207634 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:25.237512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69ffb7-0ff8-4246-b0bc-4d6e992ea393 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:25.270230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24283c-0466-4880-9272-67f4013fc982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:25.270876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24283c-0466-4880-9272-67f4013fc982 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:25.271192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24283c-0466-4880-9272-67f4013fc982 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:25.271471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24283c-0466-4880-9272-67f4013fc982 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:25.272168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24283c-0466-4880-9272-67f4013fc982 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:25.273643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24283c-0466-4880-9272-67f4013fc982 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:25.296837 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 575/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:02:25 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:02:27.319100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd651d-6b72-46dd-bb6f-566aed6d8475 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:27.319620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd651d-6b72-46dd-bb6f-566aed6d8475 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:27.319712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcd651d-6b72-46dd-bb6f-566aed6d8475 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:27.321522 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 599/1174] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:27.326895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24146457-ddda-4b5f-85d4-3f64707f3a27 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:27.327327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24146457-ddda-4b5f-85d4-3f64707f3a27 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:27.327487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24146457-ddda-4b5f-85d4-3f64707f3a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:27.344251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-24146457-ddda-4b5f-85d4-3f64707f3a27 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:27.345417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-24146457-ddda-4b5f-85d4-3f64707f3a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:27.408582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 576/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:27.433236 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd651d-6b72-46dd-bb6f-566aed6d8475 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:27.487223 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:27.531435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcd651d-6b72-46dd-bb6f-566aed6d8475 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:27.563420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f79348-74a6-4d18-924c-2f20beb8fa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:27.564258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f79348-74a6-4d18-924c-2f20beb8fa94 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:27.564540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f79348-74a6-4d18-924c-2f20beb8fa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:27.564814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f79348-74a6-4d18-924c-2f20beb8fa94 None admin] PATH_INFO: `/v3/domains/3430703476c34e1ea3010f60a470b260` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:27.565957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f79348-74a6-4d18-924c-2f20beb8fa94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3430703476c34e1ea3010f60a470b260)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:27.567399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f79348-74a6-4d18-924c-2f20beb8fa94 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:27.568970 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 600/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Tue Jul 5 15:02:27 2022] GET /identity/v3/domains/3430703476c34e1ea3010f60a470b260 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:27.575294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24146457-ddda-4b5f-85d4-3f64707f3a27 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:27.624005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:27.668549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24146457-ddda-4b5f-85d4-3f64707f3a27 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:27.701174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decd6f5c-26b1-44fa-bcce-736b226f63d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:27.701823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd6f5c-26b1-44fa-bcce-736b226f63d9 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:27.702106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd6f5c-26b1-44fa-bcce-736b226f63d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:27.702382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decd6f5c-26b1-44fa-bcce-736b226f63d9 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:27.703104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd6f5c-26b1-44fa-bcce-736b226f63d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:27.704527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decd6f5c-26b1-44fa-bcce-736b226f63d9 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:27.722755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 577/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:02:27 2022] POST /identity/v3/users => generated 311 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:02:29.730907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082bd50-5a3e-44f0-abaa-f17c99a9d9cb None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:29.731461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082bd50-5a3e-44f0-abaa-f17c99a9d9cb None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:29.731589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082bd50-5a3e-44f0-abaa-f17c99a9d9cb None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:29.732857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 601/1178] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:29.737876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12599bcf-01b1-40a3-b401-a1ecf702fc07 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:29.738265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12599bcf-01b1-40a3-b401-a1ecf702fc07 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:29.738448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12599bcf-01b1-40a3-b401-a1ecf702fc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:29.754790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-12599bcf-01b1-40a3-b401-a1ecf702fc07 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:29.756012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-12599bcf-01b1-40a3-b401-a1ecf702fc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:29.818792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 578/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:29.842375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082bd50-5a3e-44f0-abaa-f17c99a9d9cb None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:29.941460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082bd50-5a3e-44f0-abaa-f17c99a9d9cb None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:29.963105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:29.974040 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b6846-09de-479d-9a65-0b2b88897061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:29.974822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b6846-09de-479d-9a65-0b2b88897061 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:29.975137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b6846-09de-479d-9a65-0b2b88897061 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:29.975420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b6846-09de-479d-9a65-0b2b88897061 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:29.986187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b6846-09de-479d-9a65-0b2b88897061 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-c26b6846-09de-479d-9a65-0b2b88897061 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:02:29.993693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:02:29.994698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:02:29.996034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 602/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:02:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:02:30.000933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12599bcf-01b1-40a3-b401-a1ecf702fc07 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:30.040153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:30.098801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12599bcf-01b1-40a3-b401-a1ecf702fc07 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:30.130520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3876e-7668-4ef3-a544-e5b42b18ae86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:30.131233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3876e-7668-4ef3-a544-e5b42b18ae86 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:30.131522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3876e-7668-4ef3-a544-e5b42b18ae86 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:30.131801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3876e-7668-4ef3-a544-e5b42b18ae86 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:30.133090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3876e-7668-4ef3-a544-e5b42b18ae86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:30.133397 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3876e-7668-4ef3-a544-e5b42b18ae86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:30.134768 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e3876e-7668-4ef3-a544-e5b42b18ae86 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:30.142520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 579/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:02:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:30.149099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b6846-09de-479d-9a65-0b2b88897061 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:30.243960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b6846-09de-479d-9a65-0b2b88897061 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:30.277617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece13cf-00c0-42f7-9a64-81e0a97dad20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:30.278469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece13cf-00c0-42f7-9a64-81e0a97dad20 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:30.278750 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece13cf-00c0-42f7-9a64-81e0a97dad20 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:30.279025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece13cf-00c0-42f7-9a64-81e0a97dad20 None admin] PATH_INFO: `/v3/users/a193c255231642c596e7daa2b679f066` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:30.291569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece13cf-00c0-42f7-9a64-81e0a97dad20 None admin] RBAC: Authorizing `identity:get_user(user_id=a193c255231642c596e7daa2b679f066)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:30.293012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece13cf-00c0-42f7-9a64-81e0a97dad20 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:30.294618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 603/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:02:30 2022] GET /identity/v3/users/a193c255231642c596e7daa2b679f066 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:30.300401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3876e-7668-4ef3-a544-e5b42b18ae86 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:30.394424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:30.397415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3876e-7668-4ef3-a544-e5b42b18ae86 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:30.429717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdcb4d-991b-48f9-bed4-655bf3c61f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:30.430350 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdcb4d-991b-48f9-bed4-655bf3c61f00 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:30.430633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdcb4d-991b-48f9-bed4-655bf3c61f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:30.430906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdcb4d-991b-48f9-bed4-655bf3c61f00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:30.493919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdcb4d-991b-48f9-bed4-655bf3c61f00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:30.495460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdcb4d-991b-48f9-bed4-655bf3c61f00 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:30.584158 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:30.587765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 580/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:02:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:02:30.593585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece13cf-00c0-42f7-9a64-81e0a97dad20 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:30.634238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:30.694368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece13cf-00c0-42f7-9a64-81e0a97dad20 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:30.727458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834bd85-959c-4541-991b-767f4057bf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:30.728307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834bd85-959c-4541-991b-767f4057bf82 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:30.728587 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834bd85-959c-4541-991b-767f4057bf82 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:30.728859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834bd85-959c-4541-991b-767f4057bf82 None admin] PATH_INFO: `/v3/projects/18cc8ff251994e5392e5e0aeaadb5397` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:30.730009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834bd85-959c-4541-991b-767f4057bf82 None admin] RBAC: Authorizing `identity:get_project(project_id=18cc8ff251994e5392e5e0aeaadb5397)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:30.731469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834bd85-959c-4541-991b-767f4057bf82 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:30.733133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 604/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:02:30 2022] GET /identity/v3/projects/18cc8ff251994e5392e5e0aeaadb5397 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:30.739134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdcb4d-991b-48f9-bed4-655bf3c61f00 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:30.832545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdcb4d-991b-48f9-bed4-655bf3c61f00 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:30.864504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a1dfd-2626-4949-bc43-271bbe3e3a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:30.865137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a1dfd-2626-4949-bc43-271bbe3e3a01 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:30.865421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a1dfd-2626-4949-bc43-271bbe3e3a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:30.865697 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a1dfd-2626-4949-bc43-271bbe3e3a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:30.866544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a1dfd-2626-4949-bc43-271bbe3e3a01 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=18cc8ff251994e5392e5e0aeaadb5397, user.id=a193c255231642c596e7daa2b679f066) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:30.867435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a1dfd-2626-4949-bc43-271bbe3e3a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:30.868845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a1dfd-2626-4949-bc43-271bbe3e3a01 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:30.873990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 581/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:02:30 2022] GET /identity/v3/role_assignments?user.id=a193c255231642c596e7daa2b679f066&scope.project.id=18cc8ff251994e5392e5e0aeaadb5397&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:32.852641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc6bfd-2671-45fb-b137-c162e1502218 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:32.853191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc6bfd-2671-45fb-b137-c162e1502218 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:32.853191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc6bfd-2671-45fb-b137-c162e1502218 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:32.854575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 605/1186] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:32.859909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5d1af-c6e6-46fe-8ecc-91e150c735f8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:32.860301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5d1af-c6e6-46fe-8ecc-91e150c735f8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:32.860483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f5d1af-c6e6-46fe-8ecc-91e150c735f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:32.876809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f6f5d1af-c6e6-46fe-8ecc-91e150c735f8 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:32.877991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f6f5d1af-c6e6-46fe-8ecc-91e150c735f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:32.934399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:32.943018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 582/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:32.965639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc6bfd-2671-45fb-b137-c162e1502218 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:33.061941 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc6bfd-2671-45fb-b137-c162e1502218 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:33.071191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:33.094998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1a9e0-28af-43a4-b5f0-599db1b5feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:33.095805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1a9e0-28af-43a4-b5f0-599db1b5feca None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:33.096111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1a9e0-28af-43a4-b5f0-599db1b5feca None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:33.096371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1a9e0-28af-43a4-b5f0-599db1b5feca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:33.106638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1a9e0-28af-43a4-b5f0-599db1b5feca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-5cc1a9e0-28af-43a4-b5f0-599db1b5feca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:02:33.114138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:02:33.115271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:02:33.117028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 606/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:02:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:02:33.121742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5d1af-c6e6-46fe-8ecc-91e150c735f8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:33.217179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f5d1af-c6e6-46fe-8ecc-91e150c735f8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:33.250602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbf716-d70c-48cc-9376-095e33fe3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:33.251292 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf716-d70c-48cc-9376-095e33fe3f2e None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:33.251584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf716-d70c-48cc-9376-095e33fe3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:33.251865 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbf716-d70c-48cc-9376-095e33fe3f2e None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:33.252612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbf716-d70c-48cc-9376-095e33fe3f2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:33.252927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbf716-d70c-48cc-9376-095e33fe3f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:33.254309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbf716-d70c-48cc-9376-095e33fe3f2e None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:33.262621 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 583/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:02:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:33.268300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1a9e0-28af-43a4-b5f0-599db1b5feca None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:33.363541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1a9e0-28af-43a4-b5f0-599db1b5feca None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:33.395366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60f279-bf62-4642-a6fe-84bb3e6e9d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:33.396214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60f279-bf62-4642-a6fe-84bb3e6e9d90 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:33.396496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60f279-bf62-4642-a6fe-84bb3e6e9d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:33.396771 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60f279-bf62-4642-a6fe-84bb3e6e9d90 None admin] PATH_INFO: `/v3/users/a193c255231642c596e7daa2b679f066` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:33.398459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60f279-bf62-4642-a6fe-84bb3e6e9d90 None admin] RBAC: Authorizing `identity:get_user(user_id=a193c255231642c596e7daa2b679f066)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:33.399900 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60f279-bf62-4642-a6fe-84bb3e6e9d90 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:33.401475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 607/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:02:33 2022] GET /identity/v3/users/a193c255231642c596e7daa2b679f066 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:33.406978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbf716-d70c-48cc-9376-095e33fe3f2e None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:33.461680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:33.503289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbf716-d70c-48cc-9376-095e33fe3f2e None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:33.534872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5290167-0d26-4535-92fa-475f0f31281c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:33.535730 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5290167-0d26-4535-92fa-475f0f31281c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:33.536014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5290167-0d26-4535-92fa-475f0f31281c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:33.536287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5290167-0d26-4535-92fa-475f0f31281c None admin] PATH_INFO: `/v3/projects/18cc8ff251994e5392e5e0aeaadb5397` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:33.537439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5290167-0d26-4535-92fa-475f0f31281c None admin] RBAC: Authorizing `identity:get_project(project_id=18cc8ff251994e5392e5e0aeaadb5397)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:33.539004 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5290167-0d26-4535-92fa-475f0f31281c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:33.540674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 584/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:02:33 2022] GET /identity/v3/projects/18cc8ff251994e5392e5e0aeaadb5397 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:33.546358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60f279-bf62-4642-a6fe-84bb3e6e9d90 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:33.580983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:33.643721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60f279-bf62-4642-a6fe-84bb3e6e9d90 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:33.675416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f5b3fd-0148-414d-9b63-ce5278d18aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:33.676292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5b3fd-0148-414d-9b63-ce5278d18aed None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:33.676584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5b3fd-0148-414d-9b63-ce5278d18aed None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:33.676867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f5b3fd-0148-414d-9b63-ce5278d18aed None admin] PATH_INFO: `/v3/projects/18cc8ff251994e5392e5e0aeaadb5397/users/a193c255231642c596e7daa2b679f066/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:33.678814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5b3fd-0148-414d-9b63-ce5278d18aed None admin] RBAC: Authorizing `identity:create_grant(project_id=18cc8ff251994e5392e5e0aeaadb5397, user_id=a193c255231642c596e7daa2b679f066, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:33.680819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f5b3fd-0148-414d-9b63-ce5278d18aed None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:33.692617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 608/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:02:33 2022] PUT /identity/v3/projects/18cc8ff251994e5392e5e0aeaadb5397/users/a193c255231642c596e7daa2b679f066/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:02:35.670616 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e742077-43da-4b90-9b3d-bb4818ff5879 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:35.671179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e742077-43da-4b90-9b3d-bb4818ff5879 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:35.671276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e742077-43da-4b90-9b3d-bb4818ff5879 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:35.672561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 585/1193] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:02:35.677577 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2d78a-f577-411b-a593-f53f4d8ba001 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:35.677946 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2d78a-f577-411b-a593-f53f4d8ba001 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:35.678145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2d78a-f577-411b-a593-f53f4d8ba001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:35.695579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0be2d78a-f577-411b-a593-f53f4d8ba001 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:02:35.696754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0be2d78a-f577-411b-a593-f53f4d8ba001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:02:35.760660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 609/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:02:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:02:35.783115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e742077-43da-4b90-9b3d-bb4818ff5879 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:35.880137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e742077-43da-4b90-9b3d-bb4818ff5879 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:35.902455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:35.913113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45072d8-64a8-44b1-8778-73fbdc334456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:35.913913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45072d8-64a8-44b1-8778-73fbdc334456 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:35.914199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45072d8-64a8-44b1-8778-73fbdc334456 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:35.914483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45072d8-64a8-44b1-8778-73fbdc334456 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:35.924467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45072d8-64a8-44b1-8778-73fbdc334456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c45072d8-64a8-44b1-8778-73fbdc334456 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:02:35.932476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:02:35.933605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:02:35.934841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 586/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:02:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:02:35.939469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2d78a-f577-411b-a593-f53f4d8ba001 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:35.966006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:36.036591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2d78a-f577-411b-a593-f53f4d8ba001 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:36.069414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b96849-8884-44cf-9e64-0cc020fe35c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:36.070074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96849-8884-44cf-9e64-0cc020fe35c9 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:36.070364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96849-8884-44cf-9e64-0cc020fe35c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:36.070643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96849-8884-44cf-9e64-0cc020fe35c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:36.071412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b96849-8884-44cf-9e64-0cc020fe35c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:36.071724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b96849-8884-44cf-9e64-0cc020fe35c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:36.073097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b96849-8884-44cf-9e64-0cc020fe35c9 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:36.081097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 610/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:02:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:36.086761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45072d8-64a8-44b1-8778-73fbdc334456 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:36.182926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45072d8-64a8-44b1-8778-73fbdc334456 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:36.214662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc46ba-8ee4-484d-8ca3-7d165ae60265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:36.215568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc46ba-8ee4-484d-8ca3-7d165ae60265 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:36.215848 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc46ba-8ee4-484d-8ca3-7d165ae60265 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:36.216123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbc46ba-8ee4-484d-8ca3-7d165ae60265 None admin] PATH_INFO: `/v3/users/a193c255231642c596e7daa2b679f066` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:36.217287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc46ba-8ee4-484d-8ca3-7d165ae60265 None admin] RBAC: Authorizing `identity:get_user(user_id=a193c255231642c596e7daa2b679f066)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:36.218705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbc46ba-8ee4-484d-8ca3-7d165ae60265 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:36.220332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 587/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 5 15:02:36 2022] GET /identity/v3/users/a193c255231642c596e7daa2b679f066 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:36.226390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b96849-8884-44cf-9e64-0cc020fe35c9 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:36.312091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:36.324533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b96849-8884-44cf-9e64-0cc020fe35c9 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:36.356906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52609c2a-8b9f-4286-af65-f6ffbe3b350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:36.357549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609c2a-8b9f-4286-af65-f6ffbe3b350c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:36.357836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609c2a-8b9f-4286-af65-f6ffbe3b350c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:36.358116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609c2a-8b9f-4286-af65-f6ffbe3b350c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:36.422195 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52609c2a-8b9f-4286-af65-f6ffbe3b350c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:36.423664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52609c2a-8b9f-4286-af65-f6ffbe3b350c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:36.504555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:36.517170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 611/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:02:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:02:36.523713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc46ba-8ee4-484d-8ca3-7d165ae60265 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:36.573692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:36.619426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbc46ba-8ee4-484d-8ca3-7d165ae60265 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:36.650942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2da059-257f-4ade-9f19-330d68f27b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:36.651943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2da059-257f-4ade-9f19-330d68f27b35 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:36.652225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2da059-257f-4ade-9f19-330d68f27b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:36.652495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2da059-257f-4ade-9f19-330d68f27b35 None admin] PATH_INFO: `/v3/projects/18cc8ff251994e5392e5e0aeaadb5397` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:36.653647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2da059-257f-4ade-9f19-330d68f27b35 None admin] RBAC: Authorizing `identity:get_project(project_id=18cc8ff251994e5392e5e0aeaadb5397)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:36.655096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2da059-257f-4ade-9f19-330d68f27b35 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:36.656753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 588/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 5 15:02:36 2022] GET /identity/v3/projects/18cc8ff251994e5392e5e0aeaadb5397 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:36.662732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52609c2a-8b9f-4286-af65-f6ffbe3b350c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:02:36.764381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52609c2a-8b9f-4286-af65-f6ffbe3b350c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:02:36.797346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf796bef-757b-4f96-a1d3-c67f74762215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:02:36.797997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf796bef-757b-4f96-a1d3-c67f74762215 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:02:36.798287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf796bef-757b-4f96-a1d3-c67f74762215 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:02:36.798565 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf796bef-757b-4f96-a1d3-c67f74762215 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:02:36.799429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf796bef-757b-4f96-a1d3-c67f74762215 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=18cc8ff251994e5392e5e0aeaadb5397, user.id=a193c255231642c596e7daa2b679f066) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:02:36.799745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf796bef-757b-4f96-a1d3-c67f74762215 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:02:36.801177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf796bef-757b-4f96-a1d3-c67f74762215 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:02:36.806865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 612/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:02:36 2022] GET /identity/v3/role_assignments?user.id=a193c255231642c596e7daa2b679f066&scope.project.id=18cc8ff251994e5392e5e0aeaadb5397&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:02:46.646859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=40 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:02:46.804868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=74 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:31.725866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8471fa6-3aad-47cd-bd52-dd69343a8218 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:31.725866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8471fa6-3aad-47cd-bd52-dd69343a8218 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:31.725866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8471fa6-3aad-47cd-bd52-dd69343a8218 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:31.725866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 589/1201] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:03:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:03:31.731487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656ba88-e0fe-4925-a771-d1ad6cbb3bea None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:31.731911 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656ba88-e0fe-4925-a771-d1ad6cbb3bea None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:31.732092 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656ba88-e0fe-4925-a771-d1ad6cbb3bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:31.752189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-3656ba88-e0fe-4925-a771-d1ad6cbb3bea None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:03:31.753593 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3656ba88-e0fe-4925-a771-d1ad6cbb3bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:03:31.847604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 613/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:03:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:03:31.872657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8471fa6-3aad-47cd-bd52-dd69343a8218 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:32.019129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8471fa6-3aad-47cd-bd52-dd69343a8218 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:32.063839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0994f4c-3823-4a60-b82d-58d0666e3941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:32.064802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0994f4c-3823-4a60-b82d-58d0666e3941 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:32.065169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0994f4c-3823-4a60-b82d-58d0666e3941 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:32.065546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0994f4c-3823-4a60-b82d-58d0666e3941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:32.085269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0994f4c-3823-4a60-b82d-58d0666e3941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:32.087180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0994f4c-3823-4a60-b82d-58d0666e3941 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-c0994f4c-3823-4a60-b82d-58d0666e3941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:32.113797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:32.115210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:32.120462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:32.120462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:32.120462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:32.120462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:32.120462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 590/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:32.121616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656ba88-e0fe-4925-a771-d1ad6cbb3bea None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:32.248884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656ba88-e0fe-4925-a771-d1ad6cbb3bea None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:32.291478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:32.299149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3ccd6-40bb-41fb-af19-100d37250d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:32.300193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ccd6-40bb-41fb-af19-100d37250d46 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:32.300467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ccd6-40bb-41fb-af19-100d37250d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:32.300796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ccd6-40bb-41fb-af19-100d37250d46 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:32.301773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3ccd6-40bb-41fb-af19-100d37250d46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:32.302114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3ccd6-40bb-41fb-af19-100d37250d46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:32.303739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3ccd6-40bb-41fb-af19-100d37250d46 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:32.319252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 614/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:32.323179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0994f4c-3823-4a60-b82d-58d0666e3941 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:32.394416 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:32.451798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0994f4c-3823-4a60-b82d-58d0666e3941 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:32.494144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940a893-852b-45d2-b431-53db5855685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:32.494844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940a893-852b-45d2-b431-53db5855685a None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:32.495180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940a893-852b-45d2-b431-53db5855685a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:32.495437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940a893-852b-45d2-b431-53db5855685a None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:32.496280 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a940a893-852b-45d2-b431-53db5855685a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:32.497766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a940a893-852b-45d2-b431-53db5855685a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:32.520158 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 591/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 5 15:03:32 2022] POST /identity/v3/users => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:03:35.079315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98837d-5622-4c45-9638-73fccfd3d472 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:35.079882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98837d-5622-4c45-9638-73fccfd3d472 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:35.079882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98837d-5622-4c45-9638-73fccfd3d472 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:35.081638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 615/1206] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:03:35.088034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec34dc-7f61-4638-9d22-9b74121e3c43 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:35.088477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec34dc-7f61-4638-9d22-9b74121e3c43 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:35.088612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec34dc-7f61-4638-9d22-9b74121e3c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:35.106453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-93ec34dc-7f61-4638-9d22-9b74121e3c43 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:03:35.107708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-93ec34dc-7f61-4638-9d22-9b74121e3c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:03:35.181503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 592/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:03:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:03:35.206904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98837d-5622-4c45-9638-73fccfd3d472 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:35.333743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98837d-5622-4c45-9638-73fccfd3d472 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:35.373076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a57920-5df5-4653-87de-814718e3390c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:35.374148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a57920-5df5-4653-87de-814718e3390c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:35.374148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a57920-5df5-4653-87de-814718e3390c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:35.374438 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a57920-5df5-4653-87de-814718e3390c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:35.390025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a57920-5df5-4653-87de-814718e3390c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-b5a57920-5df5-4653-87de-814718e3390c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:03:35.399142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:35.400166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:35.403994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 616/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:03:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:03:35.406531 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec34dc-7f61-4638-9d22-9b74121e3c43 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:35.454042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:35.535905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec34dc-7f61-4638-9d22-9b74121e3c43 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:35.579890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d83fd-94e2-4cd8-bc92-505609310d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:35.580666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d83fd-94e2-4cd8-bc92-505609310d89 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:35.581155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d83fd-94e2-4cd8-bc92-505609310d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:35.581442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6d83fd-94e2-4cd8-bc92-505609310d89 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:35.582316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d83fd-94e2-4cd8-bc92-505609310d89 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:35.583266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d83fd-94e2-4cd8-bc92-505609310d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:35.585304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6d83fd-94e2-4cd8-bc92-505609310d89 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:35.597027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 593/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:03:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:35.603067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a57920-5df5-4653-87de-814718e3390c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:35.671424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:35.731365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a57920-5df5-4653-87de-814718e3390c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:35.767564 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d87dcd-7cf0-4307-a8b1-ebfc8e453486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:35.768402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d87dcd-7cf0-4307-a8b1-ebfc8e453486 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:35.768778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d87dcd-7cf0-4307-a8b1-ebfc8e453486 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:35.769128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d87dcd-7cf0-4307-a8b1-ebfc8e453486 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:35.781102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d87dcd-7cf0-4307-a8b1-ebfc8e453486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:35.782631 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d87dcd-7cf0-4307-a8b1-ebfc8e453486 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-25d87dcd-7cf0-4307-a8b1-ebfc8e453486 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:35.802008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:35.803275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:35.804366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:35.804366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:35.804366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:35.804366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:35.808261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 617/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:35.809756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d83fd-94e2-4cd8-bc92-505609310d89 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:35.925971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:35.930422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6d83fd-94e2-4cd8-bc92-505609310d89 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:35.976410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddace4-e45b-4ed5-a90f-a61f4b69fafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:35.977750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddace4-e45b-4ed5-a90f-a61f4b69fafb None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:35.977750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddace4-e45b-4ed5-a90f-a61f4b69fafb None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:35.977750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddace4-e45b-4ed5-a90f-a61f4b69fafb None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:35.978184 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddace4-e45b-4ed5-a90f-a61f4b69fafb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:35.978534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddace4-e45b-4ed5-a90f-a61f4b69fafb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:35.980027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddace4-e45b-4ed5-a90f-a61f4b69fafb None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:35.992470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 594/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:35.998101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d87dcd-7cf0-4307-a8b1-ebfc8e453486 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:36.113080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d87dcd-7cf0-4307-a8b1-ebfc8e453486 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:36.115526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:36.150217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59cff-6477-4e49-831e-d8cd5d90cd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:36.151091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59cff-6477-4e49-831e-d8cd5d90cd27 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:36.151341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59cff-6477-4e49-831e-d8cd5d90cd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:36.151618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded59cff-6477-4e49-831e-d8cd5d90cd27 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:36.162170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded59cff-6477-4e49-831e-d8cd5d90cd27 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:36.163666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded59cff-6477-4e49-831e-d8cd5d90cd27 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-ded59cff-6477-4e49-831e-d8cd5d90cd27 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:03:36.178573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:36.179847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:36.180807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 618/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:36.185480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddace4-e45b-4ed5-a90f-a61f4b69fafb None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:36.296006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddace4-e45b-4ed5-a90f-a61f4b69fafb None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:36.335447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f957f70-9081-4fd6-bffc-b2a01eb7a700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:36.336142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f957f70-9081-4fd6-bffc-b2a01eb7a700 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:36.336419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f957f70-9081-4fd6-bffc-b2a01eb7a700 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:36.336682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f957f70-9081-4fd6-bffc-b2a01eb7a700 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:36.337526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f957f70-9081-4fd6-bffc-b2a01eb7a700 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:36.337956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f957f70-9081-4fd6-bffc-b2a01eb7a700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:36.339415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f957f70-9081-4fd6-bffc-b2a01eb7a700 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:36.359339 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 595/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 5 15:03:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:36.365514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59cff-6477-4e49-831e-d8cd5d90cd27 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:36.476924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded59cff-6477-4e49-831e-d8cd5d90cd27 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:36.513949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296f0c-b2ad-4a50-82da-c1f3274581ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:36.514773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296f0c-b2ad-4a50-82da-c1f3274581ac None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:36.515158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296f0c-b2ad-4a50-82da-c1f3274581ac None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:36.515513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8296f0c-b2ad-4a50-82da-c1f3274581ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:36.553459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:36.598194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8296f0c-b2ad-4a50-82da-c1f3274581ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:36.600349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8296f0c-b2ad-4a50-82da-c1f3274581ac None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:36.715996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 619/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:03:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:03:36.721842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f957f70-9081-4fd6-bffc-b2a01eb7a700 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:36.742400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:36.832744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f957f70-9081-4fd6-bffc-b2a01eb7a700 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:36.878345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe49a96-82ce-4a67-bffc-ce345564286b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:36.879371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe49a96-82ce-4a67-bffc-ce345564286b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:36.880273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe49a96-82ce-4a67-bffc-ce345564286b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:36.880644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe49a96-82ce-4a67-bffc-ce345564286b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:36.893922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe49a96-82ce-4a67-bffc-ce345564286b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:36.896014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe49a96-82ce-4a67-bffc-ce345564286b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-0fe49a96-82ce-4a67-bffc-ce345564286b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:36.913780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:36.914909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:36.916033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:36.916033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:36.916033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:36.916033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:36.917613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 596/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:36.923326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296f0c-b2ad-4a50-82da-c1f3274581ac None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:36.995673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:37.037660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8296f0c-b2ad-4a50-82da-c1f3274581ac None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:37.081776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba4cd4-e4fd-4c9c-92c1-0f8086444ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:37.082694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba4cd4-e4fd-4c9c-92c1-0f8086444ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:37.083686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba4cd4-e4fd-4c9c-92c1-0f8086444ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:37.084206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba4cd4-e4fd-4c9c-92c1-0f8086444ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:37.085113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba4cd4-e4fd-4c9c-92c1-0f8086444ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:37.085547 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba4cd4-e4fd-4c9c-92c1-0f8086444ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:37.087193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba4cd4-e4fd-4c9c-92c1-0f8086444ea6 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:37.098819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 620/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:03:37.104696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe49a96-82ce-4a67-bffc-ce345564286b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:37.180205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:37.216443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe49a96-82ce-4a67-bffc-ce345564286b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:37.253535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0185fb-d233-4962-81e3-339d974602ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:37.254266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0185fb-d233-4962-81e3-339d974602ed None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:37.254610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0185fb-d233-4962-81e3-339d974602ed None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:37.254891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0185fb-d233-4962-81e3-339d974602ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:37.261818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0185fb-d233-4962-81e3-339d974602ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:37.263287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0185fb-d233-4962-81e3-339d974602ed None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-fc0185fb-d233-4962-81e3-339d974602ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:03:37.278062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:37.279105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:37.280039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:03:37.280039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:03:37.280039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:03:37.280039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:03:37.280039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:03:37.280039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:03:37.280039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:37.280039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:37.284187 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 597/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:03:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:03:37.285125 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba4cd4-e4fd-4c9c-92c1-0f8086444ea6 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:37.415753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba4cd4-e4fd-4c9c-92c1-0f8086444ea6 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:37.439790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:37.452301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e793c4e-3968-469a-9a46-41cdafee9fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:37.452963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e793c4e-3968-469a-9a46-41cdafee9fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:37.453278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e793c4e-3968-469a-9a46-41cdafee9fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:37.453523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e793c4e-3968-469a-9a46-41cdafee9fe7 None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:37.454310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e793c4e-3968-469a-9a46-41cdafee9fe7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:37.454619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e793c4e-3968-469a-9a46-41cdafee9fe7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:37.456114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e793c4e-3968-469a-9a46-41cdafee9fe7 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:37.473700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 621/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:03:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:37.475520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0185fb-d233-4962-81e3-339d974602ed None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:37.584962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0185fb-d233-4962-81e3-339d974602ed None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:37.618892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:37.626188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2b8f4-39e2-41a2-b110-c960b696fcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:37.627266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2b8f4-39e2-41a2-b110-c960b696fcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:37.627594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2b8f4-39e2-41a2-b110-c960b696fcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:37.627888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2b8f4-39e2-41a2-b110-c960b696fcb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:37.628790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2b8f4-39e2-41a2-b110-c960b696fcb5 None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=c68f1297771e4c03a897dd1f0ac21cea) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:37.629109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2b8f4-39e2-41a2-b110-c960b696fcb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:37.631225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2b8f4-39e2-41a2-b110-c960b696fcb5 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:37.647305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 598/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:03:37 2022] GET /identity/v3/role_assignments?user.id=c68f1297771e4c03a897dd1f0ac21cea&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:39.751833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66eef2c-6144-47f7-9c7e-50256d033c12 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:39.752368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66eef2c-6144-47f7-9c7e-50256d033c12 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:39.752492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66eef2c-6144-47f7-9c7e-50256d033c12 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:39.753774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 622/1220] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:03:39.759067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6778c-4c74-44f8-9dfe-a13f414b00a3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:39.759438 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6778c-4c74-44f8-9dfe-a13f414b00a3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:39.759641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6778c-4c74-44f8-9dfe-a13f414b00a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:39.776006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-00c6778c-4c74-44f8-9dfe-a13f414b00a3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:03:39.777200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-00c6778c-4c74-44f8-9dfe-a13f414b00a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:03:39.850389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 599/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:03:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:03:39.872915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66eef2c-6144-47f7-9c7e-50256d033c12 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:39.984091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66eef2c-6144-47f7-9c7e-50256d033c12 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:40.017937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab0c53-3413-4e50-98da-803c1e10e2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:40.018630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0c53-3413-4e50-98da-803c1e10e2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:40.018915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0c53-3413-4e50-98da-803c1e10e2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:40.019260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab0c53-3413-4e50-98da-803c1e10e2f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:40.030133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab0c53-3413-4e50-98da-803c1e10e2f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-dbab0c53-3413-4e50-98da-803c1e10e2f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:03:40.037869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:40.039198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:40.040408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 623/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:03:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:03:40.045423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6778c-4c74-44f8-9dfe-a13f414b00a3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:40.146447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6778c-4c74-44f8-9dfe-a13f414b00a3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:40.174176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:40.185680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e465c8-57b0-4ce6-9934-29d298ffa764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:40.186354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e465c8-57b0-4ce6-9934-29d298ffa764 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:40.186639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e465c8-57b0-4ce6-9934-29d298ffa764 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:40.186916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e465c8-57b0-4ce6-9934-29d298ffa764 None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:40.187702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e465c8-57b0-4ce6-9934-29d298ffa764 None admin] RBAC: Adding query filter params (name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:40.188010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e465c8-57b0-4ce6-9934-29d298ffa764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:40.189389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e465c8-57b0-4ce6-9934-29d298ffa764 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:40.199439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 600/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:03:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:40.205088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab0c53-3413-4e50-98da-803c1e10e2f5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:40.249802 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:40.307858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab0c53-3413-4e50-98da-803c1e10e2f5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:40.342830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b25632-4fa6-4373-a513-55c5ef26629f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:40.343706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b25632-4fa6-4373-a513-55c5ef26629f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:40.343937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b25632-4fa6-4373-a513-55c5ef26629f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:40.344218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b25632-4fa6-4373-a513-55c5ef26629f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:40.356208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b25632-4fa6-4373-a513-55c5ef26629f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:40.357678 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b25632-4fa6-4373-a513-55c5ef26629f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-52b25632-4fa6-4373-a513-55c5ef26629f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:40.375030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:40.376130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:40.377199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:40.377199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:40.377199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:40.377199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:40.377199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 624/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:40.382192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e465c8-57b0-4ce6-9934-29d298ffa764 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:40.486419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e465c8-57b0-4ce6-9934-29d298ffa764 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:40.521564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c33bf3-0b51-4984-8f93-ff8d90d9ab82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:40.522237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c33bf3-0b51-4984-8f93-ff8d90d9ab82 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:40.522513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c33bf3-0b51-4984-8f93-ff8d90d9ab82 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:40.522787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c33bf3-0b51-4984-8f93-ff8d90d9ab82 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:40.523552 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c33bf3-0b51-4984-8f93-ff8d90d9ab82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:40.523860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c33bf3-0b51-4984-8f93-ff8d90d9ab82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:40.525216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c33bf3-0b51-4984-8f93-ff8d90d9ab82 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:40.542371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 601/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:40.548487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b25632-4fa6-4373-a513-55c5ef26629f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:40.651790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:40.659456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b25632-4fa6-4373-a513-55c5ef26629f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:40.694749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb81745-89cc-4809-95d4-42af7cffd62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:40.695586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb81745-89cc-4809-95d4-42af7cffd62c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:40.695867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb81745-89cc-4809-95d4-42af7cffd62c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:40.696144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb81745-89cc-4809-95d4-42af7cffd62c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:40.704254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb81745-89cc-4809-95d4-42af7cffd62c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:40.705770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb81745-89cc-4809-95d4-42af7cffd62c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-feb81745-89cc-4809-95d4-42af7cffd62c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:03:40.719822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:40.721004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:40.722136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 625/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:40.727493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c33bf3-0b51-4984-8f93-ff8d90d9ab82 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:40.781074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:40.841923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c33bf3-0b51-4984-8f93-ff8d90d9ab82 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:40.877937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1df96-e1d5-4acd-af27-ef1ae9ceca58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:40.878877 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1df96-e1d5-4acd-af27-ef1ae9ceca58 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:40.879363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1df96-e1d5-4acd-af27-ef1ae9ceca58 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:40.880254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1df96-e1d5-4acd-af27-ef1ae9ceca58 None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:40.881453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1df96-e1d5-4acd-af27-ef1ae9ceca58 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:40.881881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1df96-e1d5-4acd-af27-ef1ae9ceca58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:40.883926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf1df96-e1d5-4acd-af27-ef1ae9ceca58 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:40.904843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 602/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 5 15:03:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:03:40.910457 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb81745-89cc-4809-95d4-42af7cffd62c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:41.018514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb81745-89cc-4809-95d4-42af7cffd62c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:41.053278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f312e05-a5f2-49d6-8cd4-ab4ffd171d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:41.054081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f312e05-a5f2-49d6-8cd4-ab4ffd171d50 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:41.054362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f312e05-a5f2-49d6-8cd4-ab4ffd171d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:41.054638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f312e05-a5f2-49d6-8cd4-ab4ffd171d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:41.065489 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f312e05-a5f2-49d6-8cd4-ab4ffd171d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:41.067033 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f312e05-a5f2-49d6-8cd4-ab4ffd171d50 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-0f312e05-a5f2-49d6-8cd4-ab4ffd171d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:41.087240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:41.088258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:41.089408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:41.089408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:41.089408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:41.089408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:41.089408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 626/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:41.094576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1df96-e1d5-4acd-af27-ef1ae9ceca58 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:41.204339 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1df96-e1d5-4acd-af27-ef1ae9ceca58 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:41.207183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:41.250034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8a426-c00c-4e49-adcc-bf85294f98e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:41.250928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8a426-c00c-4e49-adcc-bf85294f98e3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:41.251285 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8a426-c00c-4e49-adcc-bf85294f98e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:41.251653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8a426-c00c-4e49-adcc-bf85294f98e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:41.252727 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8a426-c00c-4e49-adcc-bf85294f98e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:41.253153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8a426-c00c-4e49-adcc-bf85294f98e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:41.255161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a8a426-c00c-4e49-adcc-bf85294f98e3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:41.269081 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 603/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:41.275399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f312e05-a5f2-49d6-8cd4-ab4ffd171d50 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:41.287672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:41.381559 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f312e05-a5f2-49d6-8cd4-ab4ffd171d50 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:41.416203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16125ee7-e4a9-46e5-9032-ff850b6ae248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:41.417003 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16125ee7-e4a9-46e5-9032-ff850b6ae248 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:41.417294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16125ee7-e4a9-46e5-9032-ff850b6ae248 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:41.417578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16125ee7-e4a9-46e5-9032-ff850b6ae248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:41.424228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16125ee7-e4a9-46e5-9032-ff850b6ae248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:41.425680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16125ee7-e4a9-46e5-9032-ff850b6ae248 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-16125ee7-e4a9-46e5-9032-ff850b6ae248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:03:41.440123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:41.441301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:41.443037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:03:41.443037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:03:41.443037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:03:41.443037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:03:41.443037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:03:41.443037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:03:41.443037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:41.443037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:41.443037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 627/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:03:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:03:41.448218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8a426-c00c-4e49-adcc-bf85294f98e3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:41.558489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a8a426-c00c-4e49-adcc-bf85294f98e3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:41.595409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176cdf55-90e1-412a-83c3-b77c64760a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:41.596074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cdf55-90e1-412a-83c3-b77c64760a38 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:41.596367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cdf55-90e1-412a-83c3-b77c64760a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:41.596642 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176cdf55-90e1-412a-83c3-b77c64760a38 None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:41.597435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cdf55-90e1-412a-83c3-b77c64760a38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:41.597749 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cdf55-90e1-412a-83c3-b77c64760a38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:41.599744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176cdf55-90e1-412a-83c3-b77c64760a38 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:41.612218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 604/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:03:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:41.618745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16125ee7-e4a9-46e5-9032-ff850b6ae248 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:41.696943 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:41.735392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16125ee7-e4a9-46e5-9032-ff850b6ae248 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:41.776541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc0cc7-d636-4fab-bded-0f3a2936c850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:41.777454 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc0cc7-d636-4fab-bded-0f3a2936c850 None admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:41.777720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc0cc7-d636-4fab-bded-0f3a2936c850 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:41.777993 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc0cc7-d636-4fab-bded-0f3a2936c850 None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/c68f1297771e4c03a897dd1f0ac21cea/roles/67e4864efce9469bb6bdcca8ee5d5b24` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:41.796176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc0cc7-d636-4fab-bded-0f3a2936c850 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=c68f1297771e4c03a897dd1f0ac21cea, role_id=67e4864efce9469bb6bdcca8ee5d5b24)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:41.798181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc0cc7-d636-4fab-bded-0f3a2936c850 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:41.811939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 628/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:03:41 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/c68f1297771e4c03a897dd1f0ac21cea/roles/67e4864efce9469bb6bdcca8ee5d5b24 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:03:43.879086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65348321-a6ff-4586-bd2b-62f0179d4ce2 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:43.880207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65348321-a6ff-4586-bd2b-62f0179d4ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:43.880395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65348321-a6ff-4586-bd2b-62f0179d4ce2 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:43.881892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 605/1233] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:03:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:03:43.887280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565c5df-1388-4d2c-b29f-5f597a18c1f7 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:43.887725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565c5df-1388-4d2c-b29f-5f597a18c1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:43.887852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565c5df-1388-4d2c-b29f-5f597a18c1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:43.905890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-e565c5df-1388-4d2c-b29f-5f597a18c1f7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:03:43.907459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e565c5df-1388-4d2c-b29f-5f597a18c1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:03:43.986640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 629/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:03:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:03:44.014766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65348321-a6ff-4586-bd2b-62f0179d4ce2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:44.043530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:44.135328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65348321-a6ff-4586-bd2b-62f0179d4ce2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:44.175754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e6d2e-2e6d-49d0-b820-4140bcbf3cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:44.176854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e6d2e-2e6d-49d0-b820-4140bcbf3cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:44.177216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e6d2e-2e6d-49d0-b820-4140bcbf3cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:44.177538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e6d2e-2e6d-49d0-b820-4140bcbf3cc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:44.187956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e6d2e-2e6d-49d0-b820-4140bcbf3cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-9d0e6d2e-2e6d-49d0-b820-4140bcbf3cc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:03:44.196037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:44.197282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 05 15:03:44.198355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:44.205909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 606/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 5 15:03:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:44.206076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565c5df-1388-4d2c-b29f-5f597a18c1f7 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:44.252351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:44.311421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565c5df-1388-4d2c-b29f-5f597a18c1f7 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:44.351778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5bea9-056c-46c8-8340-6695fb9c59ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:44.352454 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5bea9-056c-46c8-8340-6695fb9c59ed None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:44.352768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5bea9-056c-46c8-8340-6695fb9c59ed None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:44.353136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5bea9-056c-46c8-8340-6695fb9c59ed None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:44.354202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5bea9-056c-46c8-8340-6695fb9c59ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:44.354622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5bea9-056c-46c8-8340-6695fb9c59ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:44.356142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5bea9-056c-46c8-8340-6695fb9c59ed None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:44.365969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 630/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 5 15:03:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:44.372121 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e6d2e-2e6d-49d0-b820-4140bcbf3cc0 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:44.470208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:44.486667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e6d2e-2e6d-49d0-b820-4140bcbf3cc0 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:44.524384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b3714-8917-47db-ac3b-9d5f11ec36a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:44.525233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b3714-8917-47db-ac3b-9d5f11ec36a7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:44.525521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b3714-8917-47db-ac3b-9d5f11ec36a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:44.525803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b3714-8917-47db-ac3b-9d5f11ec36a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:44.536787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b3714-8917-47db-ac3b-9d5f11ec36a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:44.538277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b3714-8917-47db-ac3b-9d5f11ec36a7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-326b3714-8917-47db-ac3b-9d5f11ec36a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:44.555965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:44.556975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:44.558029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:44.558029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:44.558029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:44.558029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:44.558029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 607/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:03:44.563468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5bea9-056c-46c8-8340-6695fb9c59ed None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:44.686370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5bea9-056c-46c8-8340-6695fb9c59ed None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:44.698326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:44.729086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924973-da0d-450f-80d6-6791f38be558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:44.729703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924973-da0d-450f-80d6-6791f38be558 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:44.730003 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924973-da0d-450f-80d6-6791f38be558 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:44.730315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924973-da0d-450f-80d6-6791f38be558 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:44.731108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c924973-da0d-450f-80d6-6791f38be558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:44.731456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c924973-da0d-450f-80d6-6791f38be558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:44.733008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c924973-da0d-450f-80d6-6791f38be558 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:44.745663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 631/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:44.752501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b3714-8917-47db-ac3b-9d5f11ec36a7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:44.858925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b3714-8917-47db-ac3b-9d5f11ec36a7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:44.899151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7496fa3-eebe-4439-9900-6b36d9686676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:44.899896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7496fa3-eebe-4439-9900-6b36d9686676 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:44.899896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7496fa3-eebe-4439-9900-6b36d9686676 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:44.899896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7496fa3-eebe-4439-9900-6b36d9686676 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:44.902602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:44.908773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7496fa3-eebe-4439-9900-6b36d9686676 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:44.910851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7496fa3-eebe-4439-9900-6b36d9686676 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e7496fa3-eebe-4439-9900-6b36d9686676 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:03:44.927600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:44.929119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:44.930612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 608/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:44.934986 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924973-da0d-450f-80d6-6791f38be558 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:45.056809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924973-da0d-450f-80d6-6791f38be558 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:45.092069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b91a2-d029-4464-bed8-971d6610959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:45.092749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b91a2-d029-4464-bed8-971d6610959d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:45.093037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b91a2-d029-4464-bed8-971d6610959d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:45.093395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b91a2-d029-4464-bed8-971d6610959d None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:45.094314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b91a2-d029-4464-bed8-971d6610959d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:45.094802 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b91a2-d029-4464-bed8-971d6610959d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:45.096497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b91a2-d029-4464-bed8-971d6610959d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:45.116379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 632/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 5 15:03:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:45.122205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7496fa3-eebe-4439-9900-6b36d9686676 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:45.232194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7496fa3-eebe-4439-9900-6b36d9686676 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:45.275075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfe709-d6f6-4e90-ac22-08bd97e97d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:45.275721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfe709-d6f6-4e90-ac22-08bd97e97d58 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:45.276093 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfe709-d6f6-4e90-ac22-08bd97e97d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:45.276454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfe709-d6f6-4e90-ac22-08bd97e97d58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:45.342516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:45.350418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfe709-d6f6-4e90-ac22-08bd97e97d58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:45.351935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfe709-d6f6-4e90-ac22-08bd97e97d58 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:45.458007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 609/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:03:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:03:45.463897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b91a2-d029-4464-bed8-971d6610959d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:45.493665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:45.578494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b91a2-d029-4464-bed8-971d6610959d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:45.620957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb18492-996c-4a43-b940-58c00a6dad9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:45.621683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb18492-996c-4a43-b940-58c00a6dad9a None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:45.621988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb18492-996c-4a43-b940-58c00a6dad9a None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:45.622268 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb18492-996c-4a43-b940-58c00a6dad9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:45.632726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb18492-996c-4a43-b940-58c00a6dad9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:45.634271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb18492-996c-4a43-b940-58c00a6dad9a None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-7cb18492-996c-4a43-b940-58c00a6dad9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:45.653260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:45.654396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:45.655558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:45.655558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:45.655558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:45.655558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:45.655558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 633/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:45.660195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfe709-d6f6-4e90-ac22-08bd97e97d58 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:45.770686 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:45.778199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfe709-d6f6-4e90-ac22-08bd97e97d58 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:45.814000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ffbfa-3ed4-4e34-8116-85057d84a586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:45.814678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ffbfa-3ed4-4e34-8116-85057d84a586 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:45.814963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ffbfa-3ed4-4e34-8116-85057d84a586 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:45.815272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ffbfa-3ed4-4e34-8116-85057d84a586 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:45.815998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ffbfa-3ed4-4e34-8116-85057d84a586 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:45.816314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ffbfa-3ed4-4e34-8116-85057d84a586 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:45.817684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ffbfa-3ed4-4e34-8116-85057d84a586 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:45.829170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 610/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:45.834711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb18492-996c-4a43-b940-58c00a6dad9a None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:45.916487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:45.942830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb18492-996c-4a43-b940-58c00a6dad9a None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:45.995429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f53601-2e33-4eb8-bf3b-2acf6071ac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:45.996394 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f53601-2e33-4eb8-bf3b-2acf6071ac73 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:45.996669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f53601-2e33-4eb8-bf3b-2acf6071ac73 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:45.997068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f53601-2e33-4eb8-bf3b-2acf6071ac73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:46.005973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f53601-2e33-4eb8-bf3b-2acf6071ac73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:46.007955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f53601-2e33-4eb8-bf3b-2acf6071ac73 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-01f53601-2e33-4eb8-bf3b-2acf6071ac73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:03:46.026185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:46.027285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:46.033035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:03:46.033035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:03:46.033035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:03:46.033035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:03:46.033035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:03:46.033035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:03:46.033035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:46.033035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:46.033035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 634/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:03:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:46.033539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ffbfa-3ed4-4e34-8116-85057d84a586 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:46.151897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ffbfa-3ed4-4e34-8116-85057d84a586 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:46.194279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b734a-3e65-4a33-a6aa-11a7ece1fbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:46.194279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b734a-3e65-4a33-a6aa-11a7ece1fbef None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:46.194279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b734a-3e65-4a33-a6aa-11a7ece1fbef None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:46.194279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b734a-3e65-4a33-a6aa-11a7ece1fbef None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:46.194689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b734a-3e65-4a33-a6aa-11a7ece1fbef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:46.194955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b734a-3e65-4a33-a6aa-11a7ece1fbef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:46.196434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b734a-3e65-4a33-a6aa-11a7ece1fbef None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:46.211011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 611/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:03:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:03:46.216981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f53601-2e33-4eb8-bf3b-2acf6071ac73 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:46.337090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f53601-2e33-4eb8-bf3b-2acf6071ac73 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:46.371534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00bb10-8f1c-4344-a927-f06edef1f16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:46.372186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00bb10-8f1c-4344-a927-f06edef1f16f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:46.372471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00bb10-8f1c-4344-a927-f06edef1f16f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:46.372747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00bb10-8f1c-4344-a927-f06edef1f16f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:46.373593 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00bb10-8f1c-4344-a927-f06edef1f16f None admin] RBAC: Adding query filter params (role.id=67e4864efce9469bb6bdcca8ee5d5b24, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=c68f1297771e4c03a897dd1f0ac21cea) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:46.373894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00bb10-8f1c-4344-a927-f06edef1f16f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:46.375348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00bb10-8f1c-4344-a927-f06edef1f16f None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:46.381836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 635/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:03:46 2022] GET /identity/v3/role_assignments?user.id=c68f1297771e4c03a897dd1f0ac21cea&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=67e4864efce9469bb6bdcca8ee5d5b24 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:48.796905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2395a0-8c08-4467-b252-201d98ac7f48 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:48.797536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2395a0-8c08-4467-b252-201d98ac7f48 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:48.797640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2395a0-8c08-4467-b252-201d98ac7f48 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:48.799347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 612/1247] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:03:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:03:48.806289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11dd36-14ab-4d3e-a966-2c88a8578802 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:48.806732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11dd36-14ab-4d3e-a966-2c88a8578802 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:48.806861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11dd36-14ab-4d3e-a966-2c88a8578802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:48.812035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:48.826031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-dd11dd36-14ab-4d3e-a966-2c88a8578802 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:03:48.827444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-dd11dd36-14ab-4d3e-a966-2c88a8578802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:03:48.903232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 636/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:03:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:03:48.933146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2395a0-8c08-4467-b252-201d98ac7f48 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:48.956838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:49.062281 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2395a0-8c08-4467-b252-201d98ac7f48 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:49.110687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eca578-1d3a-40d6-8792-2de5028bbb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:49.111517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eca578-1d3a-40d6-8792-2de5028bbb75 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:49.111808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eca578-1d3a-40d6-8792-2de5028bbb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:49.112089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eca578-1d3a-40d6-8792-2de5028bbb75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:49.124237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eca578-1d3a-40d6-8792-2de5028bbb75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-e6eca578-1d3a-40d6-8792-2de5028bbb75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:03:49.132890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:49.134021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:49.134875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 613/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:03:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:49.140313 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11dd36-14ab-4d3e-a966-2c88a8578802 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:49.255993 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11dd36-14ab-4d3e-a966-2c88a8578802 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:49.292385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:49.297766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbaed5-ad5a-40fa-8452-79966290fab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:49.298444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbaed5-ad5a-40fa-8452-79966290fab2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:49.298733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbaed5-ad5a-40fa-8452-79966290fab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:49.299015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbaed5-ad5a-40fa-8452-79966290fab2 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:49.299798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbaed5-ad5a-40fa-8452-79966290fab2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:49.300110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbaed5-ad5a-40fa-8452-79966290fab2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:49.301514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbaed5-ad5a-40fa-8452-79966290fab2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:49.310742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 637/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:03:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:49.317263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eca578-1d3a-40d6-8792-2de5028bbb75 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:49.406495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:49.431365 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eca578-1d3a-40d6-8792-2de5028bbb75 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:49.465193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5026a3-474c-446d-9199-56044feb7a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:49.466022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5026a3-474c-446d-9199-56044feb7a64 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:49.466308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5026a3-474c-446d-9199-56044feb7a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:49.466585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5026a3-474c-446d-9199-56044feb7a64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:49.477680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5026a3-474c-446d-9199-56044feb7a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:49.479188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5026a3-474c-446d-9199-56044feb7a64 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ad5026a3-474c-446d-9199-56044feb7a64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:49.497398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:49.498719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:49.500040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:49.500040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:49.500040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:49.500040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:49.500040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 614/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:49.504623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbaed5-ad5a-40fa-8452-79966290fab2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:49.611499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbaed5-ad5a-40fa-8452-79966290fab2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:49.650125 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54518c9a-c60c-4710-83e0-186b422041e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:49.650784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54518c9a-c60c-4710-83e0-186b422041e3 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:49.651091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54518c9a-c60c-4710-83e0-186b422041e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:49.651373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54518c9a-c60c-4710-83e0-186b422041e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:49.652100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54518c9a-c60c-4710-83e0-186b422041e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:49.652408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54518c9a-c60c-4710-83e0-186b422041e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:49.653777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54518c9a-c60c-4710-83e0-186b422041e3 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:49.665990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 638/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:49.672572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5026a3-474c-446d-9199-56044feb7a64 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:49.789457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5026a3-474c-446d-9199-56044feb7a64 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:49.819245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:49.830024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2f690-0b23-4fd2-8b0a-57bd54bdf735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:49.831277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2f690-0b23-4fd2-8b0a-57bd54bdf735 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:49.831596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2f690-0b23-4fd2-8b0a-57bd54bdf735 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:49.831872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2f690-0b23-4fd2-8b0a-57bd54bdf735 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:49.842381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2f690-0b23-4fd2-8b0a-57bd54bdf735 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:49.843901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2f690-0b23-4fd2-8b0a-57bd54bdf735 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-75a2f690-0b23-4fd2-8b0a-57bd54bdf735 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:03:49.864342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:49.865464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:49.866650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 615/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:03:49.874358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54518c9a-c60c-4710-83e0-186b422041e3 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:49.935647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:49.986216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54518c9a-c60c-4710-83e0-186b422041e3 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:50.026614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb49d8-7371-4052-a1a2-071e3db449bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:50.027378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb49d8-7371-4052-a1a2-071e3db449bd None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:50.027659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb49d8-7371-4052-a1a2-071e3db449bd None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:50.027944 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb49d8-7371-4052-a1a2-071e3db449bd None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:50.028768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb49d8-7371-4052-a1a2-071e3db449bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:50.029169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb49d8-7371-4052-a1a2-071e3db449bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:50.030552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb49d8-7371-4052-a1a2-071e3db449bd None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:50.048027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 639/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 5 15:03:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:03:50.055000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2f690-0b23-4fd2-8b0a-57bd54bdf735 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:50.162841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2f690-0b23-4fd2-8b0a-57bd54bdf735 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:50.199728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc96d0-7dfd-4fa4-9040-fea5d8da7004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:50.200443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc96d0-7dfd-4fa4-9040-fea5d8da7004 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:50.200735 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc96d0-7dfd-4fa4-9040-fea5d8da7004 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:50.201018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc96d0-7dfd-4fa4-9040-fea5d8da7004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:50.269197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:50.282728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc96d0-7dfd-4fa4-9040-fea5d8da7004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:50.284302 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc96d0-7dfd-4fa4-9040-fea5d8da7004 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:50.392878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 616/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:03:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 05 15:03:50.399166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb49d8-7371-4052-a1a2-071e3db449bd None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:50.501996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb49d8-7371-4052-a1a2-071e3db449bd None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:50.515759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:50.542227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6446230-ed49-42a0-a287-8deef6dd5432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:50.543030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6446230-ed49-42a0-a287-8deef6dd5432 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:50.543338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6446230-ed49-42a0-a287-8deef6dd5432 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:50.543615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6446230-ed49-42a0-a287-8deef6dd5432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:50.554724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6446230-ed49-42a0-a287-8deef6dd5432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:50.556211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6446230-ed49-42a0-a287-8deef6dd5432 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-d6446230-ed49-42a0-a287-8deef6dd5432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:50.573528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:50.575105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:50.576166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:50.576166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:50.576166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:50.576166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:50.576166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 640/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:50.580683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc96d0-7dfd-4fa4-9040-fea5d8da7004 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:50.671454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:50.685429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc96d0-7dfd-4fa4-9040-fea5d8da7004 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:50.721974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c07dd6-2260-449b-b906-4272b98cc1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:50.722800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07dd6-2260-449b-b906-4272b98cc1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:50.723618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07dd6-2260-449b-b906-4272b98cc1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:50.723901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c07dd6-2260-449b-b906-4272b98cc1c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:50.724707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c07dd6-2260-449b-b906-4272b98cc1c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:50.725020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c07dd6-2260-449b-b906-4272b98cc1c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:50.726451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c07dd6-2260-449b-b906-4272b98cc1c7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:50.739999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 617/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:50.746151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6446230-ed49-42a0-a287-8deef6dd5432 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:50.852914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6446230-ed49-42a0-a287-8deef6dd5432 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:50.887540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ef9ba-afdd-4aca-89da-4989b30d2cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:50.888353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ef9ba-afdd-4aca-89da-4989b30d2cfe None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:50.888636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ef9ba-afdd-4aca-89da-4989b30d2cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:50.888912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ef9ba-afdd-4aca-89da-4989b30d2cfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:50.898032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ef9ba-afdd-4aca-89da-4989b30d2cfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:50.899562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454ef9ba-afdd-4aca-89da-4989b30d2cfe None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-454ef9ba-afdd-4aca-89da-4989b30d2cfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:03:50.914792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:50.915859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:50.916759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:03:50.916759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:03:50.916759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:03:50.916759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:03:50.916759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:03:50.916759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:03:50.916759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:50.916759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:50.916759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 641/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:03:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:50.922131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c07dd6-2260-449b-b906-4272b98cc1c7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:51.040148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c07dd6-2260-449b-b906-4272b98cc1c7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:51.075954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cc5d6-20a2-4d7c-951b-c9f5dd44828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:51.076631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cc5d6-20a2-4d7c-951b-c9f5dd44828d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:51.076920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cc5d6-20a2-4d7c-951b-c9f5dd44828d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:51.077199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cc5d6-20a2-4d7c-951b-c9f5dd44828d None admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:51.078011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cc5d6-20a2-4d7c-951b-c9f5dd44828d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:51.078327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cc5d6-20a2-4d7c-951b-c9f5dd44828d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:51.079835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cc5d6-20a2-4d7c-951b-c9f5dd44828d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:51.093198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 618/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:03:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:51.099887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ef9ba-afdd-4aca-89da-4989b30d2cfe None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:51.128403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:51.204677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ef9ba-afdd-4aca-89da-4989b30d2cfe None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:51.238146 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d35c5-22be-43b4-a2a1-d2723e4b71d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:51.239417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d35c5-22be-43b4-a2a1-d2723e4b71d2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:51.239698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d35c5-22be-43b4-a2a1-d2723e4b71d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:51.239988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d35c5-22be-43b4-a2a1-d2723e4b71d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:51.240805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d35c5-22be-43b4-a2a1-d2723e4b71d2 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=c68f1297771e4c03a897dd1f0ac21cea) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:51.241114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d35c5-22be-43b4-a2a1-d2723e4b71d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:51.242569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d35c5-22be-43b4-a2a1-d2723e4b71d2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:51.248155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 642/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:03:51 2022] GET /identity/v3/role_assignments?user.id=c68f1297771e4c03a897dd1f0ac21cea&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:53.282963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c8186-55f3-4182-b888-f40e3b65540f None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:53.283502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c8186-55f3-4182-b888-f40e3b65540f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:53.283641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c8186-55f3-4182-b888-f40e3b65540f None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:53.285251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 619/1261] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:03:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:03:53.290505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d10611-2fde-4007-9cff-d15766141c79 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:53.290930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d10611-2fde-4007-9cff-d15766141c79 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:53.291097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d10611-2fde-4007-9cff-d15766141c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:53.307360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-97d10611-2fde-4007-9cff-d15766141c79 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:03:53.308526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-97d10611-2fde-4007-9cff-d15766141c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:03:53.371852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 643/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:03:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:03:53.393754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c8186-55f3-4182-b888-f40e3b65540f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:53.404960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:53.491810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c8186-55f3-4182-b888-f40e3b65540f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:53.524209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5182a-b135-4bb8-8027-cd055a168a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:53.524982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5182a-b135-4bb8-8027-cd055a168a1c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:53.525263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5182a-b135-4bb8-8027-cd055a168a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:53.525538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c5182a-b135-4bb8-8027-cd055a168a1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:53.535184 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c5182a-b135-4bb8-8027-cd055a168a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-66c5182a-b135-4bb8-8027-cd055a168a1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:03:53.542547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:53.543726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:53.544794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 620/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:03:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:53.549535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d10611-2fde-4007-9cff-d15766141c79 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:53.567675 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:53.646560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d10611-2fde-4007-9cff-d15766141c79 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:53.678825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e3d58-7da1-4b26-8bc0-ed6556542847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:53.679503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e3d58-7da1-4b26-8bc0-ed6556542847 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:53.679785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e3d58-7da1-4b26-8bc0-ed6556542847 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:53.680057 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61e3d58-7da1-4b26-8bc0-ed6556542847 None admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:53.680795 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e3d58-7da1-4b26-8bc0-ed6556542847 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:53.681099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e3d58-7da1-4b26-8bc0-ed6556542847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:53.682473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61e3d58-7da1-4b26-8bc0-ed6556542847 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:53.690157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 644/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:03:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:53.695752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5182a-b135-4bb8-8027-cd055a168a1c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:53.760046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:53.792556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c5182a-b135-4bb8-8027-cd055a168a1c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:53.823902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d4013-c620-4022-81f8-8be0b709e8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:53.825063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d4013-c620-4022-81f8-8be0b709e8ba None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:53.825228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d4013-c620-4022-81f8-8be0b709e8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:53.825503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d4013-c620-4022-81f8-8be0b709e8ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:53.836304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d4013-c620-4022-81f8-8be0b709e8ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:53.837738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d4013-c620-4022-81f8-8be0b709e8ba None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-ff2d4013-c620-4022-81f8-8be0b709e8ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:53.854703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:53.855652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:53.856591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:53.856591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:53.856591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:53.856591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:53.856591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 621/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 05 15:03:53.861574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e3d58-7da1-4b26-8bc0-ed6556542847 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:53.939567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:53.960611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61e3d58-7da1-4b26-8bc0-ed6556542847 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:53.997744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c5d54-45d2-4600-af73-b65831e584e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:53.998529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c5d54-45d2-4600-af73-b65831e584e2 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:53.998875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c5d54-45d2-4600-af73-b65831e584e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:53.999241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84c5d54-45d2-4600-af73-b65831e584e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:54.000053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c5d54-45d2-4600-af73-b65831e584e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:54.000364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c5d54-45d2-4600-af73-b65831e584e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:54.001729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84c5d54-45d2-4600-af73-b65831e584e2 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:54.013738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 645/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:54.019778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d4013-c620-4022-81f8-8be0b709e8ba None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:54.115640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d4013-c620-4022-81f8-8be0b709e8ba None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:54.134587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:54.148617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59803e1b-d868-47a8-96ac-1eba574f48f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:54.149402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59803e1b-d868-47a8-96ac-1eba574f48f3 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:54.149687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59803e1b-d868-47a8-96ac-1eba574f48f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:54.149964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59803e1b-d868-47a8-96ac-1eba574f48f3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:54.157162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59803e1b-d868-47a8-96ac-1eba574f48f3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:54.158590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59803e1b-d868-47a8-96ac-1eba574f48f3 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-59803e1b-d868-47a8-96ac-1eba574f48f3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:03:54.172541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:54.173620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:54.174631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 622/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:54.179403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c5d54-45d2-4600-af73-b65831e584e2 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:54.276577 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84c5d54-45d2-4600-af73-b65831e584e2 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:54.308524 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe4597-70e4-44cc-9106-fd6106a1d493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:54.309193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4597-70e4-44cc-9106-fd6106a1d493 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:54.309480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4597-70e4-44cc-9106-fd6106a1d493 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:54.309760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe4597-70e4-44cc-9106-fd6106a1d493 None admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:54.310597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe4597-70e4-44cc-9106-fd6106a1d493 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:54.310909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe4597-70e4-44cc-9106-fd6106a1d493 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:54.312380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe4597-70e4-44cc-9106-fd6106a1d493 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:54.321181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:54.329759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 646/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 5 15:03:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:54.336109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59803e1b-d868-47a8-96ac-1eba574f48f3 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:54.431989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59803e1b-d868-47a8-96ac-1eba574f48f3 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:54.464487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf9b3-7429-4512-8dd1-70668386ed9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:54.465254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf9b3-7429-4512-8dd1-70668386ed9c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:54.465539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf9b3-7429-4512-8dd1-70668386ed9c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:54.465814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cf9b3-7429-4512-8dd1-70668386ed9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:54.475974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf9b3-7429-4512-8dd1-70668386ed9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:54.477422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cf9b3-7429-4512-8dd1-70668386ed9c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-9e4cf9b3-7429-4512-8dd1-70668386ed9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:54.494373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:54.495498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:54.496615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:54.496615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:54.496615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:54.496615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:54.496615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 623/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:54.501232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe4597-70e4-44cc-9106-fd6106a1d493 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:54.598364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe4597-70e4-44cc-9106-fd6106a1d493 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:54.631684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6acb7-796a-4d9a-add0-3d6895aff118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:54.632342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6acb7-796a-4d9a-add0-3d6895aff118 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:54.632628 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6acb7-796a-4d9a-add0-3d6895aff118 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:54.632908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6acb7-796a-4d9a-add0-3d6895aff118 None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:54.633634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6acb7-796a-4d9a-add0-3d6895aff118 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:54.633945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6acb7-796a-4d9a-add0-3d6895aff118 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:54.635341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6acb7-796a-4d9a-add0-3d6895aff118 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:54.646186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 647/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:54.651998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf9b3-7429-4512-8dd1-70668386ed9c None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:54.682474 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:54.749092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cf9b3-7429-4512-8dd1-70668386ed9c None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:54.781816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1a58d-31a2-4699-a1d2-5e0c26d71b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:54.782591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1a58d-31a2-4699-a1d2-5e0c26d71b5b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:54.782875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1a58d-31a2-4699-a1d2-5e0c26d71b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:54.783179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1a58d-31a2-4699-a1d2-5e0c26d71b5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:54.790194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1a58d-31a2-4699-a1d2-5e0c26d71b5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:54.791678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1a58d-31a2-4699-a1d2-5e0c26d71b5b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-6de1a58d-31a2-4699-a1d2-5e0c26d71b5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:03:54.805458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:54.806491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:54.807633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:03:54.807633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:03:54.807633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:03:54.807633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:03:54.807633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:03:54.807633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:03:54.807633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:54.807633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:54.807633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 624/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:03:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:54.812492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6acb7-796a-4d9a-add0-3d6895aff118 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:54.869494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:54.910887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6acb7-796a-4d9a-add0-3d6895aff118 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:54.945190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc447963-b1ad-4946-b247-bd1aecf5021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:54.945851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc447963-b1ad-4946-b247-bd1aecf5021e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:54.946145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc447963-b1ad-4946-b247-bd1aecf5021e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:54.946416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc447963-b1ad-4946-b247-bd1aecf5021e None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:54.947230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc447963-b1ad-4946-b247-bd1aecf5021e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:54.947538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc447963-b1ad-4946-b247-bd1aecf5021e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:54.949375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc447963-b1ad-4946-b247-bd1aecf5021e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:54.963296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 648/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:03:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:54.969323 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1a58d-31a2-4699-a1d2-5e0c26d71b5b None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:55.056712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:55.065746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1a58d-31a2-4699-a1d2-5e0c26d71b5b None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:55.099001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096500f-8066-4068-87db-381787b7e26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:55.099899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096500f-8066-4068-87db-381787b7e26a None admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:55.100183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096500f-8066-4068-87db-381787b7e26a None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:55.100458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096500f-8066-4068-87db-381787b7e26a None admin] PATH_INFO: `/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/c68f1297771e4c03a897dd1f0ac21cea/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:55.102373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096500f-8066-4068-87db-381787b7e26a None admin] RBAC: Authorizing `identity:create_grant(project_id=6d232251a8b245c2a547b7bc59bffb43, user_id=c68f1297771e4c03a897dd1f0ac21cea, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:55.103854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096500f-8066-4068-87db-381787b7e26a None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:55.116149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 625/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 5 15:03:54 2022] PUT /identity/v3/projects/6d232251a8b245c2a547b7bc59bffb43/users/c68f1297771e4c03a897dd1f0ac21cea/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:03:57.086456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f494-636f-431d-b122-bfe2b9c8d439 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:57.086980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f494-636f-431d-b122-bfe2b9c8d439 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:57.087090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9f494-636f-431d-b122-bfe2b9c8d439 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:57.088422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 649/1274] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:03:57.093527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b08ad-9fc8-45bf-aa90-8f568f674759 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:57.093918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b08ad-9fc8-45bf-aa90-8f568f674759 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:57.094093 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b08ad-9fc8-45bf-aa90-8f568f674759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:57.110767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b86b08ad-9fc8-45bf-aa90-8f568f674759 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:03:57.111972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b86b08ad-9fc8-45bf-aa90-8f568f674759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:03:57.176279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 626/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:03:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:03:57.199723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9f494-636f-431d-b122-bfe2b9c8d439 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:57.297120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9f494-636f-431d-b122-bfe2b9c8d439 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:57.314013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:57.330195 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fda15-2b41-4ce2-a68e-86b86a4ae22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:57.330982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fda15-2b41-4ce2-a68e-86b86a4ae22f None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:57.331292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fda15-2b41-4ce2-a68e-86b86a4ae22f None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:57.331573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fda15-2b41-4ce2-a68e-86b86a4ae22f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:57.341334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fda15-2b41-4ce2-a68e-86b86a4ae22f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-9c7fda15-2b41-4ce2-a68e-86b86a4ae22f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 05 15:03:57.349351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:57.350439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:57.351529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 650/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 5 15:03:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:57.356457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b08ad-9fc8-45bf-aa90-8f568f674759 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:57.432885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:57.450921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b08ad-9fc8-45bf-aa90-8f568f674759 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:57.483711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f36f9c-a1b7-40eb-9185-71cb98c7228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:57.484359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f36f9c-a1b7-40eb-9185-71cb98c7228d None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:57.484642 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f36f9c-a1b7-40eb-9185-71cb98c7228d None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:57.484915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f36f9c-a1b7-40eb-9185-71cb98c7228d None admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:57.485658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f36f9c-a1b7-40eb-9185-71cb98c7228d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:57.485965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f36f9c-a1b7-40eb-9185-71cb98c7228d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:57.487347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f36f9c-a1b7-40eb-9185-71cb98c7228d None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:57.494995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 627/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 5 15:03:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:57.500497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fda15-2b41-4ce2-a68e-86b86a4ae22f None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:57.595283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fda15-2b41-4ce2-a68e-86b86a4ae22f None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:57.626836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ad985-3b1a-470e-afc4-6b2dadead0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:57.627625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ad985-3b1a-470e-afc4-6b2dadead0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:57.627905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ad985-3b1a-470e-afc4-6b2dadead0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:57.628176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ad985-3b1a-470e-afc4-6b2dadead0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:57.638185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ad985-3b1a-470e-afc4-6b2dadead0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:57.639648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63ad985-3b1a-470e-afc4-6b2dadead0a7 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-a63ad985-3b1a-470e-afc4-6b2dadead0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:57.656512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:57.657583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:57.658704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:57.658704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:57.658704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:57.658704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:57.658704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 651/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:57.663246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f36f9c-a1b7-40eb-9185-71cb98c7228d None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:57.758252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f36f9c-a1b7-40eb-9185-71cb98c7228d None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:57.789720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b3fd7-ed7c-4b2a-bd63-a475287aa3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:57.790362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3fd7-ed7c-4b2a-bd63-a475287aa3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:57.790646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3fd7-ed7c-4b2a-bd63-a475287aa3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:57.790922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b3fd7-ed7c-4b2a-bd63-a475287aa3b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:57.791671 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b3fd7-ed7c-4b2a-bd63-a475287aa3b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:57.791976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b3fd7-ed7c-4b2a-bd63-a475287aa3b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:57.793314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4b3fd7-ed7c-4b2a-bd63-a475287aa3b7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:57.804983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 628/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:57.810267 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ad985-3b1a-470e-afc4-6b2dadead0a7 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:57.839672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:57.905958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ad985-3b1a-470e-afc4-6b2dadead0a7 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:57.940715 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897c48e-5e24-458e-97f3-4c1efdc09db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:57.941645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897c48e-5e24-458e-97f3-4c1efdc09db7 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:57.942058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897c48e-5e24-458e-97f3-4c1efdc09db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:57.942394 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897c48e-5e24-458e-97f3-4c1efdc09db7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:57.951668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897c48e-5e24-458e-97f3-4c1efdc09db7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:57.953471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897c48e-5e24-458e-97f3-4c1efdc09db7 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-8897c48e-5e24-458e-97f3-4c1efdc09db7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 05 15:03:57.968531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:57.969653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:03:57.970635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 652/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:57.976349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b3fd7-ed7c-4b2a-bd63-a475287aa3b7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:57.981140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:58.077882 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4b3fd7-ed7c-4b2a-bd63-a475287aa3b7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:58.110872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df391-819f-40c6-a2f7-eddeae1524af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:58.111564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df391-819f-40c6-a2f7-eddeae1524af None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:58.111850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df391-819f-40c6-a2f7-eddeae1524af None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:58.112130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df391-819f-40c6-a2f7-eddeae1524af None admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:58.112962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2df391-819f-40c6-a2f7-eddeae1524af None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:58.113849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2df391-819f-40c6-a2f7-eddeae1524af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:58.115310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2df391-819f-40c6-a2f7-eddeae1524af None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:58.131564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 629/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 5 15:03:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:58.137221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897c48e-5e24-458e-97f3-4c1efdc09db7 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:58.226219 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:58.235712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897c48e-5e24-458e-97f3-4c1efdc09db7 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:58.267691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5d296-3459-4758-9134-b4a5aa24ea92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:58.268334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5d296-3459-4758-9134-b4a5aa24ea92 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:58.268623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5d296-3459-4758-9134-b4a5aa24ea92 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:58.268907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5d296-3459-4758-9134-b4a5aa24ea92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:58.333353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5d296-3459-4758-9134-b4a5aa24ea92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:58.334810 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5d296-3459-4758-9134-b4a5aa24ea92 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:58.418442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:58.428631 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 653/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 5 15:03:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 05 15:03:58.434323 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df391-819f-40c6-a2f7-eddeae1524af None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:58.493548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:58.530547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df391-819f-40c6-a2f7-eddeae1524af None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:58.562666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db91aec-c279-4461-85f0-fef492832905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:58.563467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db91aec-c279-4461-85f0-fef492832905 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:58.563750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db91aec-c279-4461-85f0-fef492832905 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:58.564023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db91aec-c279-4461-85f0-fef492832905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:58.574373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db91aec-c279-4461-85f0-fef492832905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:58.575832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db91aec-c279-4461-85f0-fef492832905 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-4db91aec-c279-4461-85f0-fef492832905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 05 15:03:58.593046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:58.594140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:58.595149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 05 15:03:58.595149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 05 15:03:58.595149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 05 15:03:58.595149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:58.595149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 630/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:58.599727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5d296-3459-4758-9134-b4a5aa24ea92 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:58.693306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5d296-3459-4758-9134-b4a5aa24ea92 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:58.725500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472b6c5-385b-48a0-af92-d08732af4c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:58.726143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472b6c5-385b-48a0-af92-d08732af4c6c None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:58.726425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472b6c5-385b-48a0-af92-d08732af4c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:58.726700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472b6c5-385b-48a0-af92-d08732af4c6c None admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:58.727441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472b6c5-385b-48a0-af92-d08732af4c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:58.727749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472b6c5-385b-48a0-af92-d08732af4c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:58.729112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4472b6c5-385b-48a0-af92-d08732af4c6c None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:58.739770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 654/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 5 15:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:58.745227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db91aec-c279-4461-85f0-fef492832905 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:58.841901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db91aec-c279-4461-85f0-fef492832905 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:58.855683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:58.875761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c1474-74d6-42f8-ba6f-3b9f2726bfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:58.876533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c1474-74d6-42f8-ba6f-3b9f2726bfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:58.876819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c1474-74d6-42f8-ba6f-3b9f2726bfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:58.877101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c1474-74d6-42f8-ba6f-3b9f2726bfa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:58.883571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c1474-74d6-42f8-ba6f-3b9f2726bfa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:58.885003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c1474-74d6-42f8-ba6f-3b9f2726bfa7 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application [None req-d55c1474-74d6-42f8-ba6f-3b9f2726bfa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 05 15:03:58.898746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application with Lock( Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:03:58.899812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:03:58.900783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 05 15:03:58.900783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 05 15:03:58.900783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 05 15:03:58.900783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 05 15:03:58.900783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 05 15:03:58.900783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 05 15:03:58.900783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 05 15:03:58.900783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: ERROR keystone.server.flask.application Jul 05 15:03:58.900783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 631/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 5 15:03:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:03:58.905632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472b6c5-385b-48a0-af92-d08732af4c6c None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:58.965336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:03:59.013895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4472b6c5-385b-48a0-af92-d08732af4c6c None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:59.044861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b48a8c-b0de-4ffe-a23d-d240bf3c668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:59.045508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b48a8c-b0de-4ffe-a23d-d240bf3c668e None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:59.045791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b48a8c-b0de-4ffe-a23d-d240bf3c668e None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:59.046068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b48a8c-b0de-4ffe-a23d-d240bf3c668e None admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:59.046851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b48a8c-b0de-4ffe-a23d-d240bf3c668e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:59.047189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b48a8c-b0de-4ffe-a23d-d240bf3c668e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:59.048615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b48a8c-b0de-4ffe-a23d-d240bf3c668e None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:59.060607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 655/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 5 15:03:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:03:59.066539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c1474-74d6-42f8-ba6f-3b9f2726bfa7 None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:03:59.161635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c1474-74d6-42f8-ba6f-3b9f2726bfa7 None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:03:59.193816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3f6dd-2118-4176-a2a8-bf4c21d68d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:03:59.194457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f6dd-2118-4176-a2a8-bf4c21d68d80 None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:03:59.194743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f6dd-2118-4176-a2a8-bf4c21d68d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:03:59.195016 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3f6dd-2118-4176-a2a8-bf4c21d68d80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:03:59.195874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3f6dd-2118-4176-a2a8-bf4c21d68d80 None admin] RBAC: Adding query filter params (role.id=d1d68fb96ed04adf9cb22d55ba78f92f, scope.project.id=6d232251a8b245c2a547b7bc59bffb43, user.id=c68f1297771e4c03a897dd1f0ac21cea) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:03:59.196460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3f6dd-2118-4176-a2a8-bf4c21d68d80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:03:59.197592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd3f6dd-2118-4176-a2a8-bf4c21d68d80 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:03:59.203737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 632/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 5 15:03:59 2022] GET /identity/v3/role_assignments?user.id=c68f1297771e4c03a897dd1f0ac21cea&scope.project.id=6d232251a8b245c2a547b7bc59bffb43&role.id=d1d68fb96ed04adf9cb22d55ba78f92f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:04:01.205717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8f8a4-18c8-4ac1-93a8-ba5712f543a4 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:01.206156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8f8a4-18c8-4ac1-93a8-ba5712f543a4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:01.206270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8f8a4-18c8-4ac1-93a8-ba5712f543a4 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:01.208183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 656/1288] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:04:01.213131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0f8c7-707b-44be-9b26-577559d00acc None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:01.213535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0f8c7-707b-44be-9b26-577559d00acc None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:01.213691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0f8c7-707b-44be-9b26-577559d00acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:01.230403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e7d0f8c7-707b-44be-9b26-577559d00acc None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:04:01.231604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e7d0f8c7-707b-44be-9b26-577559d00acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:04:01.259577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:04:01.296480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 633/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:04:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:04:01.319777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8f8a4-18c8-4ac1-93a8-ba5712f543a4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:01.418179 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8f8a4-18c8-4ac1-93a8-ba5712f543a4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:01.428570 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:04:01.451887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655bb7-92a3-4459-9703-52f56d578fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:01.452637 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655bb7-92a3-4459-9703-52f56d578fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:01.452919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655bb7-92a3-4459-9703-52f56d578fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:01.453189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655bb7-92a3-4459-9703-52f56d578fd5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:01.453835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655bb7-92a3-4459-9703-52f56d578fd5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:01.455218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655bb7-92a3-4459-9703-52f56d578fd5 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-9a655bb7-92a3-4459-9703-52f56d578fd5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:04:01.467189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:04:01.468424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:04:01.469493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 657/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 5 15:04:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:04:01.474782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0f8c7-707b-44be-9b26-577559d00acc None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:01.568405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0f8c7-707b-44be-9b26-577559d00acc None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:01.601660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc7944-c168-4314-84a3-d49e4dc671ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:01.602312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc7944-c168-4314-84a3-d49e4dc671ce None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:01.602599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc7944-c168-4314-84a3-d49e4dc671ce None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:01.602878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc7944-c168-4314-84a3-d49e4dc671ce None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:01.603617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc7944-c168-4314-84a3-d49e4dc671ce None admin] RBAC: Adding query filter params (name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:04:01.603929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc7944-c168-4314-84a3-d49e4dc671ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:01.605289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc7944-c168-4314-84a3-d49e4dc671ce None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:01.609954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 634/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jul 5 15:04:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:04:01.616242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655bb7-92a3-4459-9703-52f56d578fd5 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:01.712083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655bb7-92a3-4459-9703-52f56d578fd5 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:01.744860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7211bdf-ff0f-41c0-bcde-ba942bc6ad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:01.745503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7211bdf-ff0f-41c0-bcde-ba942bc6ad76 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:01.745789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7211bdf-ff0f-41c0-bcde-ba942bc6ad76 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:01.746065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7211bdf-ff0f-41c0-bcde-ba942bc6ad76 None admin] PATH_INFO: `/v3/services` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:01.746770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7211bdf-ff0f-41c0-bcde-ba942bc6ad76 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:04:01.747106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7211bdf-ff0f-41c0-bcde-ba942bc6ad76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:01.748468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7211bdf-ff0f-41c0-bcde-ba942bc6ad76 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:01.752797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 658/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jul 5 15:04:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:04:03.780660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6bc2f-e62d-4f0c-8f95-34bfc7b165a1 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:03.781249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6bc2f-e62d-4f0c-8f95-34bfc7b165a1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:03.781249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6bc2f-e62d-4f0c-8f95-34bfc7b165a1 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:03.782559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 635/1293] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:04:03.788185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e55e7-62b0-4f7c-b647-1100208dbd29 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:03.788618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e55e7-62b0-4f7c-b647-1100208dbd29 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:03.788749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e55e7-62b0-4f7c-b647-1100208dbd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:03.805657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-775e55e7-62b0-4f7c-b647-1100208dbd29 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:04:03.806844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-775e55e7-62b0-4f7c-b647-1100208dbd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:04:03.835746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:04:03.876339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 659/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:04:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:04:03.899333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6bc2f-e62d-4f0c-8f95-34bfc7b165a1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:03.929888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:04:04.006204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6bc2f-e62d-4f0c-8f95-34bfc7b165a1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:04.038741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df701e8e-5505-48ff-8e1b-60bd76fded74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:04.039368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df701e8e-5505-48ff-8e1b-60bd76fded74 None admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:04.039656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df701e8e-5505-48ff-8e1b-60bd76fded74 None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:04.039935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df701e8e-5505-48ff-8e1b-60bd76fded74 None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:04.040652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df701e8e-5505-48ff-8e1b-60bd76fded74 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:04.042013 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df701e8e-5505-48ff-8e1b-60bd76fded74 None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:04.054989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 636/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 5 15:04:03 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:04:06.052757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224f16-46fc-45ad-ba6b-40dd17a33ace None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:06.053314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224f16-46fc-45ad-ba6b-40dd17a33ace None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:06.053314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05224f16-46fc-45ad-ba6b-40dd17a33ace None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:06.054694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 660/1296] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:04:06.059777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fcdd2-5395-4f88-8ed6-8594dd17b501 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:06.060214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fcdd2-5395-4f88-8ed6-8594dd17b501 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:06.060337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fcdd2-5395-4f88-8ed6-8594dd17b501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:06.077226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8d7fcdd2-5395-4f88-8ed6-8594dd17b501 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:04:06.078400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8d7fcdd2-5395-4f88-8ed6-8594dd17b501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:04:06.147560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 637/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:04:06 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 05 15:04:06.170336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224f16-46fc-45ad-ba6b-40dd17a33ace None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:06.267460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05224f16-46fc-45ad-ba6b-40dd17a33ace None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:06.299774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1fcff-5fce-4155-a567-496a6431557d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:06.300585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1fcff-5fce-4155-a567-496a6431557d None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:06.300872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1fcff-5fce-4155-a567-496a6431557d None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:06.301149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca1fcff-5fce-4155-a567-496a6431557d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:06.301799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1fcff-5fce-4155-a567-496a6431557d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:06.303248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca1fcff-5fce-4155-a567-496a6431557d None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-0ca1fcff-5fce-4155-a567-496a6431557d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:04:06.314561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:04:06.316344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:04:06.317638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 661/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 5 15:04:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:04:06.322144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fcdd2-5395-4f88-8ed6-8594dd17b501 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:06.340408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:04:06.418258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fcdd2-5395-4f88-8ed6-8594dd17b501 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:06.451088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b39557-4aa9-4fd0-bb54-84aa0bd466cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:06.451739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b39557-4aa9-4fd0-bb54-84aa0bd466cd None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:06.452022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b39557-4aa9-4fd0-bb54-84aa0bd466cd None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:06.452294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b39557-4aa9-4fd0-bb54-84aa0bd466cd None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:06.452997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b39557-4aa9-4fd0-bb54-84aa0bd466cd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:04:06.453304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b39557-4aa9-4fd0-bb54-84aa0bd466cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:06.454655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b39557-4aa9-4fd0-bb54-84aa0bd466cd None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:06.459731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 638/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jul 5 15:04:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:04:06.465493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1fcff-5fce-4155-a567-496a6431557d None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:06.496242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:04:06.562817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca1fcff-5fce-4155-a567-496a6431557d None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:06.595623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80a1a2-5626-45eb-9249-15a0e0e62d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:06.596266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a1a2-5626-45eb-9249-15a0e0e62d89 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:06.596558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a1a2-5626-45eb-9249-15a0e0e62d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:06.596843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a1a2-5626-45eb-9249-15a0e0e62d89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:06.597672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80a1a2-5626-45eb-9249-15a0e0e62d89 None admin] RBAC: Adding query filter params (interface=public, service_id=1c40477bc304450ea43e1cc5d3c88dfb, region_id=RegionOne) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:04:06.597987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80a1a2-5626-45eb-9249-15a0e0e62d89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:06.599380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80a1a2-5626-45eb-9249-15a0e0e62d89 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:06.604372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 662/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 5 15:04:06 2022] GET /identity/v3/endpoints?service_id=1c40477bc304450ea43e1cc5d3c88dfb&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:04:06.609943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b39557-4aa9-4fd0-bb54-84aa0bd466cd None admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:06.687847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:04:06.705970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b39557-4aa9-4fd0-bb54-84aa0bd466cd None admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:06.738188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e744c8-3a42-4136-bc6f-ae3350d4a93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:06.738839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e744c8-3a42-4136-bc6f-ae3350d4a93c None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:06.739160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e744c8-3a42-4136-bc6f-ae3350d4a93c None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:06.739443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e744c8-3a42-4136-bc6f-ae3350d4a93c None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:06.740121 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e744c8-3a42-4136-bc6f-ae3350d4a93c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:06.741479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e744c8-3a42-4136-bc6f-ae3350d4a93c None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:06.748035 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 639/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 5 15:04:06 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:04:08.720208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84555708-c467-4fa9-89ae-251fcedf5681 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:08.720748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84555708-c467-4fa9-89ae-251fcedf5681 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:08.720748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84555708-c467-4fa9-89ae-251fcedf5681 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:08.722155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 663/1302] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:04:08.727287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26581-fcb7-46a8-96d2-9f001ab220f2 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:08.727694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26581-fcb7-46a8-96d2-9f001ab220f2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:08.727862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26581-fcb7-46a8-96d2-9f001ab220f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:08.744492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f0f26581-fcb7-46a8-96d2-9f001ab220f2 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:04:08.745670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f0f26581-fcb7-46a8-96d2-9f001ab220f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:04:08.809626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 640/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:04:08 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 05 15:04:08.832916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84555708-c467-4fa9-89ae-251fcedf5681 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:08.924893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:04:08.933145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84555708-c467-4fa9-89ae-251fcedf5681 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:08.978691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6d8cd-2a32-4bb7-a001-bf6c8bbfe988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:08.979772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6d8cd-2a32-4bb7-a001-bf6c8bbfe988 None admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:08.980331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6d8cd-2a32-4bb7-a001-bf6c8bbfe988 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:08.980645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c6d8cd-2a32-4bb7-a001-bf6c8bbfe988 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:08.981646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6d8cd-2a32-4bb7-a001-bf6c8bbfe988 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:08.983577 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c6d8cd-2a32-4bb7-a001-bf6c8bbfe988 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application [None req-64c6d8cd-2a32-4bb7-a001-bf6c8bbfe988 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 05 15:04:08.996612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 05 15:04:08.998399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.get_or_create( Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application with Lock( Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self._enter() Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application return self.creator() Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application created_value = creator( Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: ERROR keystone.server.flask.application Jul 05 15:04:08.999670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 664/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 5 15:04:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 05 15:04:09.004357 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f26581-fcb7-46a8-96d2-9f001ab220f2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:09.079850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:04:09.102560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f26581-fcb7-46a8-96d2-9f001ab220f2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:09.135736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fdd318-3cc1-42b1-a304-76129ddeef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:09.136389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdd318-3cc1-42b1-a304-76129ddeef0b None admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:09.136675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdd318-3cc1-42b1-a304-76129ddeef0b None admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:09.136951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fdd318-3cc1-42b1-a304-76129ddeef0b None admin] PATH_INFO: `/v3/services` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:09.137661 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdd318-3cc1-42b1-a304-76129ddeef0b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:04:09.137969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdd318-3cc1-42b1-a304-76129ddeef0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:09.139433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fdd318-3cc1-42b1-a304-76129ddeef0b None admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:09.144377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 641/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jul 5 15:04:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:04:09.150072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6d8cd-2a32-4bb7-a001-bf6c8bbfe988 None admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:04:09.246921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c6d8cd-2a32-4bb7-a001-bf6c8bbfe988 None admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:04:09.278158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756dcafc-f848-46ee-83d1-3b44bd67b395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:04:09.278798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756dcafc-f848-46ee-83d1-3b44bd67b395 None admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:04:09.279108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756dcafc-f848-46ee-83d1-3b44bd67b395 None admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:04:09.279389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756dcafc-f848-46ee-83d1-3b44bd67b395 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:04:09.280132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756dcafc-f848-46ee-83d1-3b44bd67b395 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:04:09.281493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756dcafc-f848-46ee-83d1-3b44bd67b395 None admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:04:09.301203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 665/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 5 15:04:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:04:19.141519 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=30 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:04:19.296672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:11.787622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe202b-a345-4f8b-b374-e6017cac7926 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:11.788109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe202b-a345-4f8b-b374-e6017cac7926 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:11.788363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe202b-a345-4f8b-b374-e6017cac7926 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:11.789791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 642/1307] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:11.800220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5cd11-6586-4978-96dd-d3b9ecc80fdf None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:11.801717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5cd11-6586-4978-96dd-d3b9ecc80fdf None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:11.802782 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5cd11-6586-4978-96dd-d3b9ecc80fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:11.969304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-afc5cd11-6586-4978-96dd-d3b9ecc80fdf None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:11.971551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-afc5cd11-6586-4978-96dd-d3b9ecc80fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:12.095114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 666/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:12.140787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659ed9a-08e2-4977-9449-f5fd9f4205d2 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:12.141152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659ed9a-08e2-4977-9449-f5fd9f4205d2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:12.142215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659ed9a-08e2-4977-9449-f5fd9f4205d2 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:12.143711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 643/1309] 10.222.0.23 () {54 vars in 964 bytes} [Tue Jul 5 15:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:12.149783 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e879596-62f7-4802-b34c-c52791b43fb4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:12.150083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e879596-62f7-4802-b34c-c52791b43fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:12.150373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e879596-62f7-4802-b34c-c52791b43fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:12.235472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-3e879596-62f7-4802-b34c-c52791b43fb4 None None] MFA Rules not processed for user `cd7fa93466c54632ad95834297410e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:12.236685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3e879596-62f7-4802-b34c-c52791b43fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:12.258625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:12.316874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 667/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Tue Jul 5 15:05:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:12.325913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659ed9a-08e2-4977-9449-f5fd9f4205d2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:12.327681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9659ed9a-08e2-4977-9449-f5fd9f4205d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:12.440937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659ed9a-08e2-4977-9449-f5fd9f4205d2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:12.518938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b81bfa-8971-4f01-bd85-ce192c931686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:12.520005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b81bfa-8971-4f01-bd85-ce192c931686 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:12.520527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b81bfa-8971-4f01-bd85-ce192c931686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:12.521017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b81bfa-8971-4f01-bd85-ce192c931686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:12.523673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a3b81bfa-8971-4f01-bd85-ce192c931686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:12.580300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:12.637452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b81bfa-8971-4f01-bd85-ce192c931686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:12.638922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b81bfa-8971-4f01-bd85-ce192c931686 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:12.659670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 644/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:15.019475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e8bb-0bd4-4830-9d3f-dbfa8c6f5152 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:15.019994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e8bb-0bd4-4830-9d3f-dbfa8c6f5152 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:15.020114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624e8bb-0bd4-4830-9d3f-dbfa8c6f5152 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:15.022512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 668/1312] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:15.026536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcb59d-b80d-4ef3-94fd-a2ace78b5b78 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:15.026921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcb59d-b80d-4ef3-94fd-a2ace78b5b78 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:15.027149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbcb59d-b80d-4ef3-94fd-a2ace78b5b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:15.115431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8bbcb59d-b80d-4ef3-94fd-a2ace78b5b78 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:15.116689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8bbcb59d-b80d-4ef3-94fd-a2ace78b5b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:15.233334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 645/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:15.272527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e8bb-0bd4-4830-9d3f-dbfa8c6f5152 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:15.274145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2624e8bb-0bd4-4830-9d3f-dbfa8c6f5152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:15.377590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624e8bb-0bd4-4830-9d3f-dbfa8c6f5152 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:15.449974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:15.452567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e152f-5feb-4b73-bdb9-2dde97e9c038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:15.453227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e152f-5feb-4b73-bdb9-2dde97e9c038 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:15.453525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e152f-5feb-4b73-bdb9-2dde97e9c038 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:15.453806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e152f-5feb-4b73-bdb9-2dde97e9c038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:15.455931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-729e152f-5feb-4b73-bdb9-2dde97e9c038 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:15.571747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e152f-5feb-4b73-bdb9-2dde97e9c038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:15.573215 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e152f-5feb-4b73-bdb9-2dde97e9c038 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:15.592889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 669/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:17.908036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb0931-9b69-4e80-a4c5-32a9c93ca2af None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:17.908581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb0931-9b69-4e80-a4c5-32a9c93ca2af None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:17.908581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb0931-9b69-4e80-a4c5-32a9c93ca2af None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:17.909997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 646/1315] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:17.915030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782ae8d-702a-4d04-a148-dffc5176acd1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:17.915447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782ae8d-702a-4d04-a148-dffc5176acd1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:17.915627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782ae8d-702a-4d04-a148-dffc5176acd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:17.979639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b782ae8d-702a-4d04-a148-dffc5176acd1 None None] MFA Rules not processed for user `49df8493343b42b0bb70939a8aa8aee1`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:17.980851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b782ae8d-702a-4d04-a148-dffc5176acd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:18.031033 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:18.075621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 670/1316] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:18.114729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb0931-9b69-4e80-a4c5-32a9c93ca2af None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:18.116308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-98bb0931-9b69-4e80-a4c5-32a9c93ca2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:18.126404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:18.216866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb0931-9b69-4e80-a4c5-32a9c93ca2af None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:18.288068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89322408-e04c-4e82-9572-5eba2c9b8cfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:18.288701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89322408-e04c-4e82-9572-5eba2c9b8cfc service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:18.288983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89322408-e04c-4e82-9572-5eba2c9b8cfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:18.289254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89322408-e04c-4e82-9572-5eba2c9b8cfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:18.290877 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-89322408-e04c-4e82-9572-5eba2c9b8cfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:18.351090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:18.396006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89322408-e04c-4e82-9572-5eba2c9b8cfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:18.397435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89322408-e04c-4e82-9572-5eba2c9b8cfc service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:18.415770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 647/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:18 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:20.954998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc925b13-53a9-4e70-ac6f-7293254de602 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:20.955551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc925b13-53a9-4e70-ac6f-7293254de602 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:20.955643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc925b13-53a9-4e70-ac6f-7293254de602 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:20.956978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 671/1318] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:20.962304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751813e2-2e5a-42dc-88fb-60c14cb6a437 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:20.962722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751813e2-2e5a-42dc-88fb-60c14cb6a437 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:20.962868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751813e2-2e5a-42dc-88fb-60c14cb6a437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:21.025386 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-751813e2-2e5a-42dc-88fb-60c14cb6a437 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:21.026608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-751813e2-2e5a-42dc-88fb-60c14cb6a437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:21.125809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 648/1319] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:21.164120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc925b13-53a9-4e70-ac6f-7293254de602 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:21.165680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bc925b13-53a9-4e70-ac6f-7293254de602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:21.269130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc925b13-53a9-4e70-ac6f-7293254de602 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:21.338651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beef88e-c0aa-4a65-a0b4-2519d23ed2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:21.339320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beef88e-c0aa-4a65-a0b4-2519d23ed2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:21.339606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beef88e-c0aa-4a65-a0b4-2519d23ed2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:21.339881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beef88e-c0aa-4a65-a0b4-2519d23ed2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:21.341490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9beef88e-c0aa-4a65-a0b4-2519d23ed2e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:21.349151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:21.445879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beef88e-c0aa-4a65-a0b4-2519d23ed2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:21.447342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beef88e-c0aa-4a65-a0b4-2519d23ed2e6 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:21.465977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 672/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:24.141570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154b919-177d-43bf-b514-1dbdbbfb685d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:24.142114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154b919-177d-43bf-b514-1dbdbbfb685d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:24.142114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4154b919-177d-43bf-b514-1dbdbbfb685d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:24.143727 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 649/1321] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:24.149015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09decd10-0d46-4d44-9cf3-c61cf5bd7ee9 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:24.149435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09decd10-0d46-4d44-9cf3-c61cf5bd7ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:24.149598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09decd10-0d46-4d44-9cf3-c61cf5bd7ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:24.216529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-09decd10-0d46-4d44-9cf3-c61cf5bd7ee9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:24.217760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-09decd10-0d46-4d44-9cf3-c61cf5bd7ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:24.282627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:24.326270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 673/1322] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:24.363179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154b919-177d-43bf-b514-1dbdbbfb685d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:24.364753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4154b919-177d-43bf-b514-1dbdbbfb685d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:24.385583 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:24.473211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4154b919-177d-43bf-b514-1dbdbbfb685d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:24.545258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeadb3ff-a878-4622-92a3-9f5aee6cc77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:24.545903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadb3ff-a878-4622-92a3-9f5aee6cc77a service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:24.546188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadb3ff-a878-4622-92a3-9f5aee6cc77a service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:24.546465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeadb3ff-a878-4622-92a3-9f5aee6cc77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:24.548664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-aeadb3ff-a878-4622-92a3-9f5aee6cc77a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:24.620673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:24.660360 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadb3ff-a878-4622-92a3-9f5aee6cc77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:24.661702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeadb3ff-a878-4622-92a3-9f5aee6cc77a service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:24.680334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 650/1323] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:26.760324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8b09-0709-4d7c-9fad-e0629eb45d25 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:26.761184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8b09-0709-4d7c-9fad-e0629eb45d25 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:26.761474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8b09-0709-4d7c-9fad-e0629eb45d25 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:26.762899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 674/1324] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:26.768574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702f0fe-167b-42cf-8b4b-3ab0be206d74 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:26.768969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702f0fe-167b-42cf-8b4b-3ab0be206d74 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:26.769139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702f0fe-167b-42cf-8b4b-3ab0be206d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:26.832330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3702f0fe-167b-42cf-8b4b-3ab0be206d74 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:26.833565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3702f0fe-167b-42cf-8b4b-3ab0be206d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:26.928873 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 651/1325] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:26.967114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d8b09-0709-4d7c-9fad-e0629eb45d25 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:26.968641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-425d8b09-0709-4d7c-9fad-e0629eb45d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:27.072240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d8b09-0709-4d7c-9fad-e0629eb45d25 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:27.149191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0ce1c-4706-4db7-8031-b6263f6d162e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:27.149846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0ce1c-4706-4db7-8031-b6263f6d162e service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:27.150134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0ce1c-4706-4db7-8031-b6263f6d162e service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:27.150415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0ce1c-4706-4db7-8031-b6263f6d162e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:27.152101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8de0ce1c-4706-4db7-8031-b6263f6d162e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:27.169506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:27.268434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0ce1c-4706-4db7-8031-b6263f6d162e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:27.269917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0ce1c-4706-4db7-8031-b6263f6d162e service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:27.292659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 675/1326] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:29.381471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25914be3-afab-43be-9add-53cf8bcefdce None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:29.381993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25914be3-afab-43be-9add-53cf8bcefdce None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:29.382088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25914be3-afab-43be-9add-53cf8bcefdce None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:29.383602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 652/1327] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:29.389065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06753f7a-3b39-473e-b36b-17a4e9904615 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:29.389581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06753f7a-3b39-473e-b36b-17a4e9904615 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:29.389581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06753f7a-3b39-473e-b36b-17a4e9904615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:29.456125 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-06753f7a-3b39-473e-b36b-17a4e9904615 None None] MFA Rules not processed for user `49df8493343b42b0bb70939a8aa8aee1`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:29.457365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-06753f7a-3b39-473e-b36b-17a4e9904615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:29.525842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:29.559134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 676/1328] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:29.612929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513192e-dba2-460b-bfa6-bdc9b33f6f8a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:29.613220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513192e-dba2-460b-bfa6-bdc9b33f6f8a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:29.613502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8513192e-dba2-460b-bfa6-bdc9b33f6f8a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:29.614895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 653/1329] 10.222.0.23 () {54 vars in 964 bytes} [Tue Jul 5 15:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:29.621020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cee55-7419-4cd0-b76e-b07e7ca27985 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:29.621377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cee55-7419-4cd0-b76e-b07e7ca27985 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:29.621721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cee55-7419-4cd0-b76e-b07e7ca27985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:29.697132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-646cee55-7419-4cd0-b76e-b07e7ca27985 None None] MFA Rules not processed for user `cd7fa93466c54632ad95834297410e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:29.698666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-646cee55-7419-4cd0-b76e-b07e7ca27985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:29.775869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 677/1330] 10.222.0.23 () {58 vars in 1061 bytes} [Tue Jul 5 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:05:29.784770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513192e-dba2-460b-bfa6-bdc9b33f6f8a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:29.786302 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8513192e-dba2-460b-bfa6-bdc9b33f6f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:29.817203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:29.898700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8513192e-dba2-460b-bfa6-bdc9b33f6f8a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:29.977866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a911bef0-1374-4aea-b57d-0238b3b55725 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:29.978525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911bef0-1374-4aea-b57d-0238b3b55725 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:29.978815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911bef0-1374-4aea-b57d-0238b3b55725 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:29.979155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a911bef0-1374-4aea-b57d-0238b3b55725 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:29.980842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a911bef0-1374-4aea-b57d-0238b3b55725 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:30.056694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:30.089264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911bef0-1374-4aea-b57d-0238b3b55725 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:30.090712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a911bef0-1374-4aea-b57d-0238b3b55725 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:30.108946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 654/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:29 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:32.732905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23213868-df32-4c62-aff4-92dd5b9c669a None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:32.733458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23213868-df32-4c62-aff4-92dd5b9c669a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:32.733458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23213868-df32-4c62-aff4-92dd5b9c669a None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:32.734855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 678/1332] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:32.740112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a0d64-18af-4cd0-9847-acb2d270eee7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:32.740505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a0d64-18af-4cd0-9847-acb2d270eee7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:32.740759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a0d64-18af-4cd0-9847-acb2d270eee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:32.803284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-921a0d64-18af-4cd0-9847-acb2d270eee7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:32.804483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-921a0d64-18af-4cd0-9847-acb2d270eee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:32.898057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 655/1333] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:32.933368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23213868-df32-4c62-aff4-92dd5b9c669a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:32.934885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-23213868-df32-4c62-aff4-92dd5b9c669a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:33.004165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:33.037632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23213868-df32-4c62-aff4-92dd5b9c669a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:33.108334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c17511-3137-4deb-92a1-ebfb355b1fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:33.108987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c17511-3137-4deb-92a1-ebfb355b1fae service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:33.109275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c17511-3137-4deb-92a1-ebfb355b1fae service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:33.110100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c17511-3137-4deb-92a1-ebfb355b1fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:33.111799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-56c17511-3137-4deb-92a1-ebfb355b1fae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:33.217877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c17511-3137-4deb-92a1-ebfb355b1fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:33.219910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c17511-3137-4deb-92a1-ebfb355b1fae service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:33.229753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:33.238811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 679/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:35.287833 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ff00e-5c2d-4d0b-bd94-0cbfa4acc622 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:35.288440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ff00e-5c2d-4d0b-bd94-0cbfa4acc622 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:35.288440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ff00e-5c2d-4d0b-bd94-0cbfa4acc622 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:35.289782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 656/1335] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:35.294746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31e52c-bea6-4c90-af18-05a5bb512d45 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:35.295199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31e52c-bea6-4c90-af18-05a5bb512d45 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:35.295346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31e52c-bea6-4c90-af18-05a5bb512d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:35.364027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-9a31e52c-bea6-4c90-af18-05a5bb512d45 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:35.365238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9a31e52c-bea6-4c90-af18-05a5bb512d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:35.458053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 680/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:35.496014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ff00e-5c2d-4d0b-bd94-0cbfa4acc622 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:35.497599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ba9ff00e-5c2d-4d0b-bd94-0cbfa4acc622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:35.537535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:35.599320 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ff00e-5c2d-4d0b-bd94-0cbfa4acc622 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:35.670576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad72766-ebd3-4c55-a984-6549e795c175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:35.671240 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad72766-ebd3-4c55-a984-6549e795c175 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:35.671523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad72766-ebd3-4c55-a984-6549e795c175 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:35.672322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad72766-ebd3-4c55-a984-6549e795c175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:35.673932 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fad72766-ebd3-4c55-a984-6549e795c175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:35.756311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:35.782543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad72766-ebd3-4c55-a984-6549e795c175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:35.784031 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad72766-ebd3-4c55-a984-6549e795c175 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:35.802119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 657/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:38.248965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c93e8f-fa6e-4da0-998c-98904fa51479 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:38.249463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c93e8f-fa6e-4da0-998c-98904fa51479 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:38.249567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c93e8f-fa6e-4da0-998c-98904fa51479 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:38.251546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 681/1338] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:38.256612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9857f5a-ef81-4be5-b2fe-ef146dcc6e54 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:38.257083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9857f5a-ef81-4be5-b2fe-ef146dcc6e54 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:38.257181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9857f5a-ef81-4be5-b2fe-ef146dcc6e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:38.319962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e9857f5a-ef81-4be5-b2fe-ef146dcc6e54 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:38.321171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e9857f5a-ef81-4be5-b2fe-ef146dcc6e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:38.415747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 658/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:38 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:38.451441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c93e8f-fa6e-4da0-998c-98904fa51479 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:38.453495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-99c93e8f-fa6e-4da0-998c-98904fa51479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:38.525787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:38.554158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c93e8f-fa6e-4da0-998c-98904fa51479 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:38.627775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f306d34-e760-461a-af0a-2496ffd937c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:38.628429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f306d34-e760-461a-af0a-2496ffd937c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:38.628709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f306d34-e760-461a-af0a-2496ffd937c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:38.628990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f306d34-e760-461a-af0a-2496ffd937c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:38.630634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3f306d34-e760-461a-af0a-2496ffd937c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:38.738155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f306d34-e760-461a-af0a-2496ffd937c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:38.739563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f306d34-e760-461a-af0a-2496ffd937c7 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:38.757752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 682/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:38 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:42.815120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b719f-cbe8-42d5-904b-859e18e17867 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:42.815625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b719f-cbe8-42d5-904b-859e18e17867 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:42.815625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b719f-cbe8-42d5-904b-859e18e17867 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:42.817182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 659/1341] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:42.822166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6551fd8-eb61-4e10-9cc4-e18707a2af1c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:42.822583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6551fd8-eb61-4e10-9cc4-e18707a2af1c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:42.822740 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6551fd8-eb61-4e10-9cc4-e18707a2af1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:42.829235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:42.893621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-e6551fd8-eb61-4e10-9cc4-e18707a2af1c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:42.894839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e6551fd8-eb61-4e10-9cc4-e18707a2af1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:43.013629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 683/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:43.052889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b719f-cbe8-42d5-904b-859e18e17867 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:43.054987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-150b719f-cbe8-42d5-904b-859e18e17867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:43.104006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:43.149251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b719f-cbe8-42d5-904b-859e18e17867 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:43.152781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a66391-3b2f-448b-8770-8ea35ed292ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:43.153419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a66391-3b2f-448b-8770-8ea35ed292ba service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:43.153704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a66391-3b2f-448b-8770-8ea35ed292ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:43.153980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a66391-3b2f-448b-8770-8ea35ed292ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:43.163423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a66391-3b2f-448b-8770-8ea35ed292ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:43.164858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a66391-3b2f-448b-8770-8ea35ed292ba service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:43.174169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 660/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:45.822057 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc377e-e648-4ca8-8b6e-6e7f4e62deaa None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:45.822567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc377e-e648-4ca8-8b6e-6e7f4e62deaa None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:45.822567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc377e-e648-4ca8-8b6e-6e7f4e62deaa None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:45.824223 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 684/1344] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:45.830149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5da05c-5f37-45cc-ad80-64c74e27bff4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:45.830580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5da05c-5f37-45cc-ad80-64c74e27bff4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:45.830719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5da05c-5f37-45cc-ad80-64c74e27bff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:45.858105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4e5da05c-5f37-45cc-ad80-64c74e27bff4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:45.859329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4e5da05c-5f37-45cc-ad80-64c74e27bff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:45.943792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 661/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:45.983324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc377e-e648-4ca8-8b6e-6e7f4e62deaa None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:45.985232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-fafc377e-e648-4ca8-8b6e-6e7f4e62deaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:45.998853 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc377e-e648-4ca8-8b6e-6e7f4e62deaa None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:46.003000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e98a46-095f-4526-a0cc-74575385031a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:46.003685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e98a46-095f-4526-a0cc-74575385031a service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:46.003975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e98a46-095f-4526-a0cc-74575385031a service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:46.004255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e98a46-095f-4526-a0cc-74575385031a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:46.014221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e98a46-095f-4526-a0cc-74575385031a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:46.016287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e98a46-095f-4526-a0cc-74575385031a service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:46.026592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 685/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:49.976438 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519a45-8b14-43c2-880f-4e1a82a5887a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:49.976965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519a45-8b14-43c2-880f-4e1a82a5887a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:49.977048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2519a45-8b14-43c2-880f-4e1a82a5887a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:49.978395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 662/1347] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:49.984538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cada449-38b3-4228-80cf-2364df577dc9 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:49.985409 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cada449-38b3-4228-80cf-2364df577dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:49.985409 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cada449-38b3-4228-80cf-2364df577dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:50.004344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4cada449-38b3-4228-80cf-2364df577dc9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:50.005568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4cada449-38b3-4228-80cf-2364df577dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:50.013815 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 686/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:50.051308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2519a45-8b14-43c2-880f-4e1a82a5887a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:05:50.061876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2519a45-8b14-43c2-880f-4e1a82a5887a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:05:50.065880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439e279-0a53-4ba7-ac43-cc1b2ccad5bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:05:50.066524 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439e279-0a53-4ba7-ac43-cc1b2ccad5bd service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:50.066812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439e279-0a53-4ba7-ac43-cc1b2ccad5bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:50.067119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439e279-0a53-4ba7-ac43-cc1b2ccad5bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:50.076289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439e279-0a53-4ba7-ac43-cc1b2ccad5bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:05:50.077729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439e279-0a53-4ba7-ac43-cc1b2ccad5bd service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:05:50.087214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 663/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:05:55.067722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec1326-22df-4d7f-bfb9-3f20aa66c949 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:55.068156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec1326-22df-4d7f-bfb9-3f20aa66c949 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:55.068355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec1326-22df-4d7f-bfb9-3f20aa66c949 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:55.069658 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 687/1350] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:55.075401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9922ead-0bc6-4497-84a2-4fcf54c4374a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:55.076398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9922ead-0bc6-4497-84a2-4fcf54c4374a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:55.076689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9922ead-0bc6-4497-84a2-4fcf54c4374a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:55.177775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e9922ead-0bc6-4497-84a2-4fcf54c4374a None None] MFA Rules not processed for user `c68f1297771e4c03a897dd1f0ac21cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:55.178979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e9922ead-0bc6-4497-84a2-4fcf54c4374a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:55.219138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:05:55.266856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 664/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:05:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:59.425394 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2771cb1-dcae-457f-af50-0b7769c4180a None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:59.425957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2771cb1-dcae-457f-af50-0b7769c4180a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:59.426281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2771cb1-dcae-457f-af50-0b7769c4180a None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:59.427976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 688/1352] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:05:59.432867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe2263-09ab-4dbc-a70a-3f681c497a27 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:05:59.433285 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe2263-09ab-4dbc-a70a-3f681c497a27 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:05:59.433427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe2263-09ab-4dbc-a70a-3f681c497a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:05:59.453381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5ebe2263-09ab-4dbc-a70a-3f681c497a27 None None] MFA Rules not processed for user `c68f1297771e4c03a897dd1f0ac21cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:05:59.454491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5ebe2263-09ab-4dbc-a70a-3f681c497a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:05:59.463208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 665/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:05:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:05:59.998464 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=72 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:06:00.986431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e888-d960-4ebd-804d-ff8f591b0ff8 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:00.987270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e888-d960-4ebd-804d-ff8f591b0ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:00.987399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e888-d960-4ebd-804d-ff8f591b0ff8 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:00.989142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 689/1354] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:06:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:00.994227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d050b-55e0-4971-bc80-271122f77941 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:00.994645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d050b-55e0-4971-bc80-271122f77941 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:00.994788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d050b-55e0-4971-bc80-271122f77941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:01.013252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-004d050b-55e0-4971-bc80-271122f77941 None None] MFA Rules not processed for user `c68f1297771e4c03a897dd1f0ac21cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:01.015185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-004d050b-55e0-4971-bc80-271122f77941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:01.023898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 666/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:06:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:04.705774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274170e-7cca-45ef-99c5-b46abdba34bc None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:04.706266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274170e-7cca-45ef-99c5-b46abdba34bc None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:04.706266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274170e-7cca-45ef-99c5-b46abdba34bc None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:04.707867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 690/1356] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:04.714354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1090d4-b982-4a72-a018-1f8adea62317 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:04.715162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1090d4-b982-4a72-a018-1f8adea62317 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:04.715630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1090d4-b982-4a72-a018-1f8adea62317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:04.737312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-bc1090d4-b982-4a72-a018-1f8adea62317 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:04.739505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bc1090d4-b982-4a72-a018-1f8adea62317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:04.752129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 667/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:04.808919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d3ea0-a9bc-43c9-b676-43aa5b8a1d3c None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:04.809540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d3ea0-a9bc-43c9-b676-43aa5b8a1d3c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:04.810206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d3ea0-a9bc-43c9-b676-43aa5b8a1d3c None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:04.812333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 691/1358] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jul 5 15:06:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:04.819130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40567cb-09ab-4e3a-b7fe-ae941d3f3f63 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:04.819972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40567cb-09ab-4e3a-b7fe-ae941d3f3f63 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:04.820643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40567cb-09ab-4e3a-b7fe-ae941d3f3f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:04.875033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c40567cb-09ab-4e3a-b7fe-ae941d3f3f63 None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:04.876426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c40567cb-09ab-4e3a-b7fe-ae941d3f3f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:04.975108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 668/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jul 5 15:06:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:06:04.983851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d3ea0-a9bc-43c9-b676-43aa5b8a1d3c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:04.999824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d3ea0-a9bc-43c9-b676-43aa5b8a1d3c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:05.005592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e4276-bae5-4606-844d-14f367dee958 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:05.006381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e4276-bae5-4606-844d-14f367dee958 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:05.006689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e4276-bae5-4606-844d-14f367dee958 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:05.007126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e4276-bae5-4606-844d-14f367dee958 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:05.016380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e4276-bae5-4606-844d-14f367dee958 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:05.017967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e4276-bae5-4606-844d-14f367dee958 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:05.038024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 692/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:07.253651 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18a293-b677-4a13-8957-212fd6692206 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:07.254126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18a293-b677-4a13-8957-212fd6692206 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:07.254217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18a293-b677-4a13-8957-212fd6692206 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:07.255740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 669/1361] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:07.260637 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec2418-9f61-4f40-aeb9-5d1bd82fd6aa None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:07.261045 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec2418-9f61-4f40-aeb9-5d1bd82fd6aa None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:07.261200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec2418-9f61-4f40-aeb9-5d1bd82fd6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:07.279217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-e3ec2418-9f61-4f40-aeb9-5d1bd82fd6aa None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:07.280686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e3ec2418-9f61-4f40-aeb9-5d1bd82fd6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:07.288783 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 693/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:07.333165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18a293-b677-4a13-8957-212fd6692206 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:07.348195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18a293-b677-4a13-8957-212fd6692206 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:07.353869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3d960-36dc-4602-8b25-7f034b2b1898 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:07.354729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3d960-36dc-4602-8b25-7f034b2b1898 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:07.355235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3d960-36dc-4602-8b25-7f034b2b1898 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:07.355698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3d960-36dc-4602-8b25-7f034b2b1898 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:07.364927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3d960-36dc-4602-8b25-7f034b2b1898 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:07.366893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3d960-36dc-4602-8b25-7f034b2b1898 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:07.378196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 670/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:09.723159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25f16a-ebe0-4c3c-aadf-334020fb4308 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:09.724133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25f16a-ebe0-4c3c-aadf-334020fb4308 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:09.724682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25f16a-ebe0-4c3c-aadf-334020fb4308 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:09.726575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 694/1364] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:09.732063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aabc5-e5b8-42ee-a615-bebe42b558a9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:09.732498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aabc5-e5b8-42ee-a615-bebe42b558a9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:09.732757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45aabc5-e5b8-42ee-a615-bebe42b558a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:09.764319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e45aabc5-e5b8-42ee-a615-bebe42b558a9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:09.766353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e45aabc5-e5b8-42ee-a615-bebe42b558a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:09.777562 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 671/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:09.822851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2853-87e0-46c6-adda-e62653d779c1 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:09.823428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2853-87e0-46c6-adda-e62653d779c1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:09.823715 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a2853-87e0-46c6-adda-e62653d779c1 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:09.825095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 695/1366] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jul 5 15:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:09.831282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e746a-643e-4a1b-8652-e805b9d6e14c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:09.831571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e746a-643e-4a1b-8652-e805b9d6e14c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:09.831850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e746a-643e-4a1b-8652-e805b9d6e14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:09.849666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-835e746a-643e-4a1b-8652-e805b9d6e14c None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:09.850905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-835e746a-643e-4a1b-8652-e805b9d6e14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:09.860029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 672/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jul 5 15:06:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:09.867786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a2853-87e0-46c6-adda-e62653d779c1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:09.882511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a2853-87e0-46c6-adda-e62653d779c1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:09.888036 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a5ad5-7fd2-4620-8115-228211f70249 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:09.888673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a5ad5-7fd2-4620-8115-228211f70249 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:09.888959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a5ad5-7fd2-4620-8115-228211f70249 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:09.889837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a5ad5-7fd2-4620-8115-228211f70249 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:09.898048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a5ad5-7fd2-4620-8115-228211f70249 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:09.899721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a5ad5-7fd2-4620-8115-228211f70249 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:09.909118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 696/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:12.007024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86468240-37da-47ab-9d18-17d65c97cae9 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:12.007509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86468240-37da-47ab-9d18-17d65c97cae9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:12.007627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86468240-37da-47ab-9d18-17d65c97cae9 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:12.009002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 673/1369] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:12.018175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad780487-9367-40f1-bce8-d5a682a28d04 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:12.018648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad780487-9367-40f1-bce8-d5a682a28d04 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:12.019101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad780487-9367-40f1-bce8-d5a682a28d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:12.037160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ad780487-9367-40f1-bce8-d5a682a28d04 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:12.038589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ad780487-9367-40f1-bce8-d5a682a28d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:12.047131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 697/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:12.092570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86468240-37da-47ab-9d18-17d65c97cae9 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:12.107737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86468240-37da-47ab-9d18-17d65c97cae9 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:12.113556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d936c-1bcc-49ca-9000-572c5b675bd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:12.114313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d936c-1bcc-49ca-9000-572c5b675bd4 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:12.114709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d936c-1bcc-49ca-9000-572c5b675bd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:12.115135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d936c-1bcc-49ca-9000-572c5b675bd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:12.124369 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d936c-1bcc-49ca-9000-572c5b675bd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:12.126339 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d936c-1bcc-49ca-9000-572c5b675bd4 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:12.135994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 674/1371] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:12.756791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176742db-c396-4faa-8731-65e6e18d660a None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:12.757081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176742db-c396-4faa-8731-65e6e18d660a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:12.757573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176742db-c396-4faa-8731-65e6e18d660a None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:12.759138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 698/1372] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:12.764478 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121d1cb-c501-4d8e-b49b-3b794d645e55 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:12.764773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121d1cb-c501-4d8e-b49b-3b794d645e55 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:12.765060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121d1cb-c501-4d8e-b49b-3b794d645e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:12.783262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e121d1cb-c501-4d8e-b49b-3b794d645e55 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:12.784473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e121d1cb-c501-4d8e-b49b-3b794d645e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:12.792611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 675/1373] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:13.129822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0caa08-e985-4917-9b49-506a303f7d7f None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:13.130356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0caa08-e985-4917-9b49-506a303f7d7f None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:13.130520 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0caa08-e985-4917-9b49-506a303f7d7f None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:13.131957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 699/1374] 10.222.0.23 () {54 vars in 963 bytes} [Tue Jul 5 15:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:13.136930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb046a-e98e-4156-9c0a-3941785cb0e3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:13.137381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb046a-e98e-4156-9c0a-3941785cb0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:13.137722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb046a-e98e-4156-9c0a-3941785cb0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:13.145929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:06:13.196850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7acb046a-e98e-4156-9c0a-3941785cb0e3 None None] MFA Rules not processed for user `8ad7cee9ffed415b9a30176b1888e562`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:13.204772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7acb046a-e98e-4156-9c0a-3941785cb0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:13.269517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 676/1375] 10.222.0.23 () {58 vars in 1060 bytes} [Tue Jul 5 15:06:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:06:13.277903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0caa08-e985-4917-9b49-506a303f7d7f None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:13.289627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0caa08-e985-4917-9b49-506a303f7d7f None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:13.293419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c30f9c-d06d-4ed8-8b36-ab61daafb16f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:13.294066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c30f9c-d06d-4ed8-8b36-ab61daafb16f service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:13.294349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c30f9c-d06d-4ed8-8b36-ab61daafb16f service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:13.294624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c30f9c-d06d-4ed8-8b36-ab61daafb16f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:13.303827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c30f9c-d06d-4ed8-8b36-ab61daafb16f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:13.305258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c30f9c-d06d-4ed8-8b36-ab61daafb16f service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:13.315176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 700/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:14.225748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94f460-d19b-4ac6-9dc7-8e7a1dcdabe6 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:14.226209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94f460-d19b-4ac6-9dc7-8e7a1dcdabe6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:14.226438 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94f460-d19b-4ac6-9dc7-8e7a1dcdabe6 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:14.228179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 677/1377] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:14.233404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75da6b-38d8-4873-b3d8-0f3a087f890a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:14.233912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75da6b-38d8-4873-b3d8-0f3a087f890a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:14.235439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d75da6b-38d8-4873-b3d8-0f3a087f890a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:14.252205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0d75da6b-38d8-4873-b3d8-0f3a087f890a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:14.253395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0d75da6b-38d8-4873-b3d8-0f3a087f890a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:14.261440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 701/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:14.306383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94f460-d19b-4ac6-9dc7-8e7a1dcdabe6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:14.321757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94f460-d19b-4ac6-9dc7-8e7a1dcdabe6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:14.328112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3a4b8-5ba7-4f7e-9f7e-ca96b9a3d913 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:14.328749 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3a4b8-5ba7-4f7e-9f7e-ca96b9a3d913 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:14.329037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3a4b8-5ba7-4f7e-9f7e-ca96b9a3d913 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:14.329316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3a4b8-5ba7-4f7e-9f7e-ca96b9a3d913 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:14.337941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3a4b8-5ba7-4f7e-9f7e-ca96b9a3d913 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:14.339563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3a4b8-5ba7-4f7e-9f7e-ca96b9a3d913 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:14.349140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 678/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:06:16.396912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d2553-c94a-48e8-9dc4-305379e50255 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:16.397432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d2553-c94a-48e8-9dc4-305379e50255 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:16.397547 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d2553-c94a-48e8-9dc4-305379e50255 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:16.398866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 702/1380] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:16.404099 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab626c-f206-4bab-ba35-e96405fc3bc6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:16.404541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab626c-f206-4bab-ba35-e96405fc3bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:16.405280 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab626c-f206-4bab-ba35-e96405fc3bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:16.422977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3eab626c-f206-4bab-ba35-e96405fc3bc6 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:16.424195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3eab626c-f206-4bab-ba35-e96405fc3bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:16.432032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 679/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:16.475932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d2553-c94a-48e8-9dc4-305379e50255 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:16.490065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d2553-c94a-48e8-9dc4-305379e50255 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:16.495462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f4a28-7a9c-4a87-b670-c6b7f966dcf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:16.496111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f4a28-7a9c-4a87-b670-c6b7f966dcf7 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:16.496396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f4a28-7a9c-4a87-b670-c6b7f966dcf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:16.496657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f4a28-7a9c-4a87-b670-c6b7f966dcf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:16.505072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f4a28-7a9c-4a87-b670-c6b7f966dcf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:16.506645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f4a28-7a9c-4a87-b670-c6b7f966dcf7 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:16.516395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 703/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:18.591471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be19ed1-9e3b-45bc-a444-bc2855a59618 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:18.592199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be19ed1-9e3b-45bc-a444-bc2855a59618 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:18.592199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be19ed1-9e3b-45bc-a444-bc2855a59618 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:18.593554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 680/1383] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:18.598885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4613c0-178b-463d-acdc-412d6faccbcc None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:18.599332 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4613c0-178b-463d-acdc-412d6faccbcc None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:18.599506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4613c0-178b-463d-acdc-412d6faccbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:18.618481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-3e4613c0-178b-463d-acdc-412d6faccbcc None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:18.620192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3e4613c0-178b-463d-acdc-412d6faccbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:18.629509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 704/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:06:18.672143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be19ed1-9e3b-45bc-a444-bc2855a59618 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:18.687390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be19ed1-9e3b-45bc-a444-bc2855a59618 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:18.693079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0722315d-0ad9-4835-9cb9-674f08afc512 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:18.693858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722315d-0ad9-4835-9cb9-674f08afc512 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:18.694240 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722315d-0ad9-4835-9cb9-674f08afc512 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:18.694617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0722315d-0ad9-4835-9cb9-674f08afc512 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:18.703987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722315d-0ad9-4835-9cb9-674f08afc512 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:18.705674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0722315d-0ad9-4835-9cb9-674f08afc512 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:18.715905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 681/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:20.775246 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931c869-7d9f-4ed9-bb62-ff7a66c85a28 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:20.775748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931c869-7d9f-4ed9-bb62-ff7a66c85a28 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:20.775748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931c869-7d9f-4ed9-bb62-ff7a66c85a28 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:20.777100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 705/1386] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:20.783204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba6a09-5d6d-4c80-b7ac-d82533da3338 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:20.783999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba6a09-5d6d-4c80-b7ac-d82533da3338 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:20.784437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba6a09-5d6d-4c80-b7ac-d82533da3338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:20.804505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-05ba6a09-5d6d-4c80-b7ac-d82533da3338 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:20.805709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-05ba6a09-5d6d-4c80-b7ac-d82533da3338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:20.815401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 682/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:20.860854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931c869-7d9f-4ed9-bb62-ff7a66c85a28 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:20.876333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931c869-7d9f-4ed9-bb62-ff7a66c85a28 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:20.882366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4801d08-5a96-47cc-ab93-7b6227eb2f97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:20.883026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4801d08-5a96-47cc-ab93-7b6227eb2f97 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:20.883351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4801d08-5a96-47cc-ab93-7b6227eb2f97 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:20.883631 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4801d08-5a96-47cc-ab93-7b6227eb2f97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:20.892103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4801d08-5a96-47cc-ab93-7b6227eb2f97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:20.893702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4801d08-5a96-47cc-ab93-7b6227eb2f97 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:20.903266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 706/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:22.985865 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a4499-fe22-43e8-8943-7b1d0cfc2857 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:22.986562 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a4499-fe22-43e8-8943-7b1d0cfc2857 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:22.986562 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a4499-fe22-43e8-8943-7b1d0cfc2857 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:22.987885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 683/1389] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:22.993222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb100d6-f34b-43ab-955c-0131be3af7f0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:22.993732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb100d6-f34b-43ab-955c-0131be3af7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:22.993732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb100d6-f34b-43ab-955c-0131be3af7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:23.012341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-abb100d6-f34b-43ab-955c-0131be3af7f0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:23.013574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-abb100d6-f34b-43ab-955c-0131be3af7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:23.021319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 707/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:06:23.066298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a4499-fe22-43e8-8943-7b1d0cfc2857 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:23.085746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a4499-fe22-43e8-8943-7b1d0cfc2857 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:23.093770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c97d9-3f38-45f4-8118-3dfefae772ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:23.094568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c97d9-3f38-45f4-8118-3dfefae772ee service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:23.094963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c97d9-3f38-45f4-8118-3dfefae772ee service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:23.095404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c97d9-3f38-45f4-8118-3dfefae772ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:23.106059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c97d9-3f38-45f4-8118-3dfefae772ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:23.108222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c97d9-3f38-45f4-8118-3dfefae772ee service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:23.120644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 684/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:25.221981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea254b-8a5b-4d39-9271-21cc3cbc9b6f None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:25.222530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea254b-8a5b-4d39-9271-21cc3cbc9b6f None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:25.222530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea254b-8a5b-4d39-9271-21cc3cbc9b6f None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:25.224033 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 708/1392] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:25.229653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358561dd-bfe9-45df-b1f9-e754b67be02e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:25.230264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358561dd-bfe9-45df-b1f9-e754b67be02e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:25.230264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358561dd-bfe9-45df-b1f9-e754b67be02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:25.250440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-358561dd-bfe9-45df-b1f9-e754b67be02e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:25.251677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-358561dd-bfe9-45df-b1f9-e754b67be02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:25.259777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 685/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:25.306061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea254b-8a5b-4d39-9271-21cc3cbc9b6f None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:25.321045 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea254b-8a5b-4d39-9271-21cc3cbc9b6f None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:25.327066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1127acbc-5170-4e94-a8ce-0036164472d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:25.327733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127acbc-5170-4e94-a8ce-0036164472d7 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:25.328064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127acbc-5170-4e94-a8ce-0036164472d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:25.328317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1127acbc-5170-4e94-a8ce-0036164472d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:25.336696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127acbc-5170-4e94-a8ce-0036164472d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:25.338291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1127acbc-5170-4e94-a8ce-0036164472d7 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:25.347765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 709/1394] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:27.413601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf26135-140d-43bd-bcc1-6c190d9f9982 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:27.414123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf26135-140d-43bd-bcc1-6c190d9f9982 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:27.414243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf26135-140d-43bd-bcc1-6c190d9f9982 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:27.415579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 686/1395] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:27.421348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b242f74-6c58-4ccc-a109-0b6ac7e088c2 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:27.421753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b242f74-6c58-4ccc-a109-0b6ac7e088c2 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:27.422012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b242f74-6c58-4ccc-a109-0b6ac7e088c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:27.440275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-2b242f74-6c58-4ccc-a109-0b6ac7e088c2 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:27.441477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2b242f74-6c58-4ccc-a109-0b6ac7e088c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:27.449703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 710/1396] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:27.493079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf26135-140d-43bd-bcc1-6c190d9f9982 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:27.507903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf26135-140d-43bd-bcc1-6c190d9f9982 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:27.513257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3372052c-c075-44cb-8c4d-ec2dc9c1cb9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:27.513926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372052c-c075-44cb-8c4d-ec2dc9c1cb9c service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:27.514221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372052c-c075-44cb-8c4d-ec2dc9c1cb9c service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:27.514504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3372052c-c075-44cb-8c4d-ec2dc9c1cb9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:27.523059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372052c-c075-44cb-8c4d-ec2dc9c1cb9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:27.524672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3372052c-c075-44cb-8c4d-ec2dc9c1cb9c service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:27.534092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 687/1397] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:28.472070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19034b7-2cb0-423f-94e5-1305aad46c7d None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:28.472576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19034b7-2cb0-423f-94e5-1305aad46c7d None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:28.472660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19034b7-2cb0-423f-94e5-1305aad46c7d None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:28.473989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 711/1398] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:28.479245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab576-fe0c-4188-bd0a-25e2342c3295 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:28.479659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab576-fe0c-4188-bd0a-25e2342c3295 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:28.479802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aab576-fe0c-4188-bd0a-25e2342c3295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:28.497148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-18aab576-fe0c-4188-bd0a-25e2342c3295 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:28.498856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-18aab576-fe0c-4188-bd0a-25e2342c3295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:28.506656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 688/1399] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:28.562115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30adb-1a2b-4410-adb4-ee09d485df13 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:28.562403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30adb-1a2b-4410-adb4-ee09d485df13 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:28.562685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30adb-1a2b-4410-adb4-ee09d485df13 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:28.564161 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 712/1400] 10.222.0.23 () {54 vars in 962 bytes} [Tue Jul 5 15:06:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:28.569256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ef91b-ea0a-4acd-8db1-e38b17d0d3f6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:28.569627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ef91b-ea0a-4acd-8db1-e38b17d0d3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:28.569975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5ef91b-ea0a-4acd-8db1-e38b17d0d3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:28.616869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9d5ef91b-ea0a-4acd-8db1-e38b17d0d3f6 None None] MFA Rules not processed for user `450bcd0f22b840c299d08ce9433d6c85`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:28.618048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9d5ef91b-ea0a-4acd-8db1-e38b17d0d3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:28.653520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:06:28.683165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 689/1401] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Jul 5 15:06:28 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:28.691219 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae30adb-1a2b-4410-adb4-ee09d485df13 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:28.702620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae30adb-1a2b-4410-adb4-ee09d485df13 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:28.706118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398eb522-9ceb-4028-8aca-880e1a6991cd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450bcd0f22b840c299d08ce9433d6c85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:28.706754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eb522-9ceb-4028-8aca-880e1a6991cd service glance] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:28.707055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eb522-9ceb-4028-8aca-880e1a6991cd service glance] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:28.707345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398eb522-9ceb-4028-8aca-880e1a6991cd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:28.716316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398eb522-9ceb-4028-8aca-880e1a6991cd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:28.717748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398eb522-9ceb-4028-8aca-880e1a6991cd service glance] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:28.727121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 713/1402] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:29.433252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc9f5c-2b37-4339-9f4f-a780df26a17f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:29.433545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc9f5c-2b37-4339-9f4f-a780df26a17f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:29.433829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc9f5c-2b37-4339-9f4f-a780df26a17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:29.482257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b9bc9f5c-2b37-4339-9f4f-a780df26a17f None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:29.483600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b9bc9f5c-2b37-4339-9f4f-a780df26a17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:29.547023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 690/1403] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:29.590628 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d219-1d1f-4576-8308-229479a8c355 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:29.591157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d219-1d1f-4576-8308-229479a8c355 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:29.591497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751d219-1d1f-4576-8308-229479a8c355 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:29.591630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b246461-9de4-49c6-b2a8-db16085c9dc9 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:29.591855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b246461-9de4-49c6-b2a8-db16085c9dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:29.592139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b246461-9de4-49c6-b2a8-db16085c9dc9 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:29.593061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 714/1404] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:06:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:29.593485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 691/1405] 10.222.0.23 () {54 vars in 942 bytes} [Tue Jul 5 15:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:06:29.598051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03b2ca-e7b5-4338-98c6-10cd14921d33 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:29.598347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03b2ca-e7b5-4338-98c6-10cd14921d33 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:29.598745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03b2ca-e7b5-4338-98c6-10cd14921d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:29.599590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8266b-75e9-443a-a2d1-4928dd366042 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:29.599884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8266b-75e9-443a-a2d1-4928dd366042 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:29.600167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8266b-75e9-443a-a2d1-4928dd366042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:29.616689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-cb03b2ca-e7b5-4338-98c6-10cd14921d33 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:29.617849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-cb03b2ca-e7b5-4338-98c6-10cd14921d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:29.625756 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 715/1406] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:29.647460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8aa8266b-75e9-443a-a2d1-4928dd366042 None None] MFA Rules not processed for user `4c73aa1f244d4251b8539c2806f38a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:29.648659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8aa8266b-75e9-443a-a2d1-4928dd366042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:29.671266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03b2ca-e7b5-4338-98c6-10cd14921d33 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:29.683857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03b2ca-e7b5-4338-98c6-10cd14921d33 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:29.689301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a42be7-3c2a-4201-ac1d-fc24163f362e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:29.689945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a42be7-3c2a-4201-ac1d-fc24163f362e service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:29.690235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a42be7-3c2a-4201-ac1d-fc24163f362e service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:29.690516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a42be7-3c2a-4201-ac1d-fc24163f362e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:29.698869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a42be7-3c2a-4201-ac1d-fc24163f362e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:29.700499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a42be7-3c2a-4201-ac1d-fc24163f362e service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:29.710263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 716/1407] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:29.722195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 692/1408] 10.222.0.23 () {58 vars in 1039 bytes} [Tue Jul 5 15:06:29 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:29.732252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a42be7-3c2a-4201-ac1d-fc24163f362e service nova] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:29.743377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a42be7-3c2a-4201-ac1d-fc24163f362e service nova] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:29.748371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a7ba1-0c49-44c0-b145-f26c3cae6d48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:29.749025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a7ba1-0c49-44c0-b145-f26c3cae6d48 service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:29.749308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a7ba1-0c49-44c0-b145-f26c3cae6d48 service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:29.749584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a7ba1-0c49-44c0-b145-f26c3cae6d48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:29.758324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a7ba1-0c49-44c0-b145-f26c3cae6d48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:29.759954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a7ba1-0c49-44c0-b145-f26c3cae6d48 service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:29.769200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 717/1409] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 05 15:06:39.719249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=94 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:06:39.765478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=75 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:06:51.730591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f3a92-2e72-4058-bf19-654c4e0ac9ab None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:51.731204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f3a92-2e72-4058-bf19-654c4e0ac9ab None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:51.731204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920f3a92-2e72-4058-bf19-654c4e0ac9ab None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:51.732582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 693/1410] 10.222.0.23 () {54 vars in 918 bytes} [Tue Jul 5 15:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:51.737747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7db87-8644-4338-b88c-cb4b119db77a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:51.738209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7db87-8644-4338-b88c-cb4b119db77a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:51.738315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7db87-8644-4338-b88c-cb4b119db77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:51.756053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-99a7db87-8644-4338-b88c-cb4b119db77a None None] MFA Rules not processed for user `c68f1297771e4c03a897dd1f0ac21cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:51.757235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-99a7db87-8644-4338-b88c-cb4b119db77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:51.766151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 718/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Jul 5 15:06:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:51.779920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68870c34-a7a4-4aa6-a968-2901a595850a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:51.780211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68870c34-a7a4-4aa6-a968-2901a595850a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:51.780492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68870c34-a7a4-4aa6-a968-2901a595850a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:51.781811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 694/1412] 10.222.0.23 () {54 vars in 918 bytes} [Tue Jul 5 15:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:51.786051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971746d-053e-42b0-bc99-2641b92fd3bd None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:51.786339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971746d-053e-42b0-bc99-2641b92fd3bd None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:51.786618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971746d-053e-42b0-bc99-2641b92fd3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:51.803695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-e971746d-053e-42b0-bc99-2641b92fd3bd None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:51.804848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e971746d-053e-42b0-bc99-2641b92fd3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:51.814281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 719/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Jul 5 15:06:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:54.282966 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381a9b-9de4-4816-8b8b-207796543b8d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:54.283551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381a9b-9de4-4816-8b8b-207796543b8d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:54.283649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88381a9b-9de4-4816-8b8b-207796543b8d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:54.285001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 695/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:54.290867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43daa8-5b5f-41e0-aa6d-33e9b6af34ad None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:54.290867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43daa8-5b5f-41e0-aa6d-33e9b6af34ad None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:54.291458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43daa8-5b5f-41e0-aa6d-33e9b6af34ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:54.309013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ea43daa8-5b5f-41e0-aa6d-33e9b6af34ad None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:54.310700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ea43daa8-5b5f-41e0-aa6d-33e9b6af34ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:54.318317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 720/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:54.362704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88381a9b-9de4-4816-8b8b-207796543b8d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:54.377762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88381a9b-9de4-4816-8b8b-207796543b8d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:54.383390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c30405-c7d2-445e-930a-15d4902a8b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:54.384034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c30405-c7d2-445e-930a-15d4902a8b64 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:54.384347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c30405-c7d2-445e-930a-15d4902a8b64 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:54.384627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c30405-c7d2-445e-930a-15d4902a8b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:54.392775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c30405-c7d2-445e-930a-15d4902a8b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:54.394354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c30405-c7d2-445e-930a-15d4902a8b64 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:54.404022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 696/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:55.298655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eb884-4a78-4557-b193-b369885a0610 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:55.299229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eb884-4a78-4557-b193-b369885a0610 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:55.299229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eb884-4a78-4557-b193-b369885a0610 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:55.300625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 721/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:55.305551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6782586-2e78-4d7a-b945-fcf698ba685f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:55.305964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6782586-2e78-4d7a-b945-fcf698ba685f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:55.306115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6782586-2e78-4d7a-b945-fcf698ba685f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:55.323702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e6782586-2e78-4d7a-b945-fcf698ba685f None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:55.325170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e6782586-2e78-4d7a-b945-fcf698ba685f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:55.333426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 697/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:55.482549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eb884-4a78-4557-b193-b369885a0610 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:55.496592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eb884-4a78-4557-b193-b369885a0610 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:55.502507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b4693-6182-4b13-b32e-4ba4b7f0c637 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:55.503181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b4693-6182-4b13-b32e-4ba4b7f0c637 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:55.503475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b4693-6182-4b13-b32e-4ba4b7f0c637 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:55.503754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b4693-6182-4b13-b32e-4ba4b7f0c637 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:55.511873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b4693-6182-4b13-b32e-4ba4b7f0c637 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:55.514089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b4693-6182-4b13-b32e-4ba4b7f0c637 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:55.523557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 722/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:56.410622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a776e-3569-4614-bfd9-d06f17b919ef None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:56.411206 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a776e-3569-4614-bfd9-d06f17b919ef None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:56.411206 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a776e-3569-4614-bfd9-d06f17b919ef None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:56.412709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 698/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:56.417510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324c5c2-f78f-4fc1-a7be-2a2a3a56b7e9 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:56.417911 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324c5c2-f78f-4fc1-a7be-2a2a3a56b7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:56.418083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324c5c2-f78f-4fc1-a7be-2a2a3a56b7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:56.436604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-f324c5c2-f78f-4fc1-a7be-2a2a3a56b7e9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:56.437802 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f324c5c2-f78f-4fc1-a7be-2a2a3a56b7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:56.445472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 723/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:56.596049 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a776e-3569-4614-bfd9-d06f17b919ef None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:56.610691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a776e-3569-4614-bfd9-d06f17b919ef None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:56.616032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619689a-ddc0-4021-8a24-8413eeb593c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:56.616660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619689a-ddc0-4021-8a24-8413eeb593c6 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:56.616946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619689a-ddc0-4021-8a24-8413eeb593c6 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:56.617225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619689a-ddc0-4021-8a24-8413eeb593c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:56.625144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619689a-ddc0-4021-8a24-8413eeb593c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:56.626758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f619689a-ddc0-4021-8a24-8413eeb593c6 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:56.636044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 699/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:57.487416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115089fa-ff0a-4063-a293-a40b5351c506 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:57.487947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115089fa-ff0a-4063-a293-a40b5351c506 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:57.487947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115089fa-ff0a-4063-a293-a40b5351c506 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:57.489287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 724/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:57.494148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e57591-f22d-4e20-a20d-343c34433c24 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:57.494581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e57591-f22d-4e20-a20d-343c34433c24 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:57.494722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e57591-f22d-4e20-a20d-343c34433c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:57.513162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-92e57591-f22d-4e20-a20d-343c34433c24 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:57.514448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-92e57591-f22d-4e20-a20d-343c34433c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:57.522463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 700/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:57.672312 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115089fa-ff0a-4063-a293-a40b5351c506 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:57.686834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115089fa-ff0a-4063-a293-a40b5351c506 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:57.692104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3506a37c-2ca3-4729-a7f0-c8e7f222996c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:57.692733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506a37c-2ca3-4729-a7f0-c8e7f222996c service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:57.693018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506a37c-2ca3-4729-a7f0-c8e7f222996c service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:57.693296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506a37c-2ca3-4729-a7f0-c8e7f222996c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:57.701445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506a37c-2ca3-4729-a7f0-c8e7f222996c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:57.703014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3506a37c-2ca3-4729-a7f0-c8e7f222996c service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:57.712243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 725/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:58.558417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e69a3f-e0cd-4dfd-824a-00e63baa368f None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:58.558954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e69a3f-e0cd-4dfd-824a-00e63baa368f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:58.558954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e69a3f-e0cd-4dfd-824a-00e63baa368f None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:58.560282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 701/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:58.565290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2166a-40b7-46ca-8cb6-94b589fe5ccd None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:58.565677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2166a-40b7-46ca-8cb6-94b589fe5ccd None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:58.565896 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2166a-40b7-46ca-8cb6-94b589fe5ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:58.584025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-68b2166a-40b7-46ca-8cb6-94b589fe5ccd None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:58.585241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-68b2166a-40b7-46ca-8cb6-94b589fe5ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:58.593801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 726/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:58.743886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e69a3f-e0cd-4dfd-824a-00e63baa368f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:58.758137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e69a3f-e0cd-4dfd-824a-00e63baa368f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:58.763571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6159c-162a-4a58-ac5c-3796d5acf350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:58.764207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6159c-162a-4a58-ac5c-3796d5acf350 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:58.764495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6159c-162a-4a58-ac5c-3796d5acf350 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:58.764777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6159c-162a-4a58-ac5c-3796d5acf350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:58.772821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6159c-162a-4a58-ac5c-3796d5acf350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:58.774410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6159c-162a-4a58-ac5c-3796d5acf350 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:58.784206 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 702/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:06:59.582748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d992e1-483d-44a4-9fec-181e60216ea8 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:59.583177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d992e1-483d-44a4-9fec-181e60216ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:59.583332 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d992e1-483d-44a4-9fec-181e60216ea8 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:59.584736 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 727/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:06:59.589791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511790c7-1056-470c-a3c7-2d45983e2240 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:59.590208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511790c7-1056-470c-a3c7-2d45983e2240 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:59.590352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511790c7-1056-470c-a3c7-2d45983e2240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:59.608955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-511790c7-1056-470c-a3c7-2d45983e2240 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:06:59.610149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-511790c7-1056-470c-a3c7-2d45983e2240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:06:59.617807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 703/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:06:59.766877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d992e1-483d-44a4-9fec-181e60216ea8 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:06:59.781642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d992e1-483d-44a4-9fec-181e60216ea8 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:06:59.786945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8a0d3-deca-45eb-8dd9-1b4e1e749dea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:06:59.787608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8a0d3-deca-45eb-8dd9-1b4e1e749dea service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:06:59.787904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8a0d3-deca-45eb-8dd9-1b4e1e749dea service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:06:59.788183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8a0d3-deca-45eb-8dd9-1b4e1e749dea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:06:59.796177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8a0d3-deca-45eb-8dd9-1b4e1e749dea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:06:59.797758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8a0d3-deca-45eb-8dd9-1b4e1e749dea service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:06:59.806976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 728/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:00.626942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e835866-95c7-44c2-9b89-9a7aa0bb3603 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:00.627513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e835866-95c7-44c2-9b89-9a7aa0bb3603 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:00.627513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e835866-95c7-44c2-9b89-9a7aa0bb3603 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:00.628892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 704/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:00.633855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618b5c3-a948-40aa-87e1-ce7b55fd85a1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:00.634253 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618b5c3-a948-40aa-87e1-ce7b55fd85a1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:00.634421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0618b5c3-a948-40aa-87e1-ce7b55fd85a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:00.652392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0618b5c3-a948-40aa-87e1-ce7b55fd85a1 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:00.653581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0618b5c3-a948-40aa-87e1-ce7b55fd85a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:00.661886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 729/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:00.812981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e835866-95c7-44c2-9b89-9a7aa0bb3603 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:00.827714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e835866-95c7-44c2-9b89-9a7aa0bb3603 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:00.832968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99da5d6-7287-47c3-8df3-2cf8bd9a552a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:00.833598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99da5d6-7287-47c3-8df3-2cf8bd9a552a service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:00.833882 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99da5d6-7287-47c3-8df3-2cf8bd9a552a service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:00.834164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99da5d6-7287-47c3-8df3-2cf8bd9a552a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:00.842067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99da5d6-7287-47c3-8df3-2cf8bd9a552a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:00.843679 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99da5d6-7287-47c3-8df3-2cf8bd9a552a service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:00.852784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 705/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:01.653926 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eec2e-622a-4add-a839-4d1623f1762e None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:01.654979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eec2e-622a-4add-a839-4d1623f1762e None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:01.655106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6eec2e-622a-4add-a839-4d1623f1762e None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:01.656421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 730/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:01.661478 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2336c6-9592-4191-90a1-16012b97b296 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:01.661910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2336c6-9592-4191-90a1-16012b97b296 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:01.662054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2336c6-9592-4191-90a1-16012b97b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:01.680218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2f2336c6-9592-4191-90a1-16012b97b296 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:01.681418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2f2336c6-9592-4191-90a1-16012b97b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:01.689352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 706/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:01.837500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eec2e-622a-4add-a839-4d1623f1762e None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:01.852186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6eec2e-622a-4add-a839-4d1623f1762e None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:01.858164 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a8844-782c-47a6-a3e4-f750f880bd1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:01.858794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a8844-782c-47a6-a3e4-f750f880bd1f service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:01.859107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a8844-782c-47a6-a3e4-f750f880bd1f service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:01.859388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a8844-782c-47a6-a3e4-f750f880bd1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:01.867290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a8844-782c-47a6-a3e4-f750f880bd1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:01.868858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a8844-782c-47a6-a3e4-f750f880bd1f service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:01.878296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 731/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:02.676343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e916cf-0a39-437a-ba2a-ddf82737ce9d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:02.676904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e916cf-0a39-437a-ba2a-ddf82737ce9d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:02.676904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e916cf-0a39-437a-ba2a-ddf82737ce9d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:02.678266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 707/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:02.683061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4e309-6e18-41b1-84c1-8b90bce1ff96 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:02.683504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4e309-6e18-41b1-84c1-8b90bce1ff96 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:02.683657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4e309-6e18-41b1-84c1-8b90bce1ff96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:02.702126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-03c4e309-6e18-41b1-84c1-8b90bce1ff96 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:02.703356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-03c4e309-6e18-41b1-84c1-8b90bce1ff96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:02.711079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 732/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:02.859748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e916cf-0a39-437a-ba2a-ddf82737ce9d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:02.873816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e916cf-0a39-437a-ba2a-ddf82737ce9d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:02.879200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44aa11-52d1-4744-9fd3-19ac50c6deb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:02.879836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44aa11-52d1-4744-9fd3-19ac50c6deb2 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:02.880126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44aa11-52d1-4744-9fd3-19ac50c6deb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:02.880407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44aa11-52d1-4744-9fd3-19ac50c6deb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:02.888357 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44aa11-52d1-4744-9fd3-19ac50c6deb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:02.890453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44aa11-52d1-4744-9fd3-19ac50c6deb2 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:02.899741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 708/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:04.809829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88e4f8-486a-47ca-b401-0578b5668324 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:04.810418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88e4f8-486a-47ca-b401-0578b5668324 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:04.810418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88e4f8-486a-47ca-b401-0578b5668324 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:04.812442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 733/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:07:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:04.817530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36775ab9-3c4e-4a5b-972a-93f8574790f4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:04.817977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36775ab9-3c4e-4a5b-972a-93f8574790f4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:04.818098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36775ab9-3c4e-4a5b-972a-93f8574790f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:04.836354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-36775ab9-3c4e-4a5b-972a-93f8574790f4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:04.837554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-36775ab9-3c4e-4a5b-972a-93f8574790f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:04.845435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 709/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:04.883615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88e4f8-486a-47ca-b401-0578b5668324 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:04.894396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88e4f8-486a-47ca-b401-0578b5668324 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:04.898059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10b3f5-c36e-44d7-8be1-7b20a758137e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:04.898697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10b3f5-c36e-44d7-8be1-7b20a758137e service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:04.898982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10b3f5-c36e-44d7-8be1-7b20a758137e service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:04.899290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10b3f5-c36e-44d7-8be1-7b20a758137e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:04.908234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10b3f5-c36e-44d7-8be1-7b20a758137e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:04.909670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10b3f5-c36e-44d7-8be1-7b20a758137e service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:04.918928 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 734/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:07.121706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00021cf4-976a-4721-b1bf-44be1e683051 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:07.122301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00021cf4-976a-4721-b1bf-44be1e683051 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:07.122301 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00021cf4-976a-4721-b1bf-44be1e683051 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:07.123719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 710/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:07.129061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082a409-2737-421f-8155-93b7ee2fb3c4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:07.129480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082a409-2737-421f-8155-93b7ee2fb3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:07.129625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0082a409-2737-421f-8155-93b7ee2fb3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:07.147689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0082a409-2737-421f-8155-93b7ee2fb3c4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:07.148884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0082a409-2737-421f-8155-93b7ee2fb3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:07.157335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 735/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:07.194029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00021cf4-976a-4721-b1bf-44be1e683051 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:07.205719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00021cf4-976a-4721-b1bf-44be1e683051 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:07.209332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f63eb9-6caf-4f23-9b61-acb169ca7145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:07.209974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f63eb9-6caf-4f23-9b61-acb169ca7145 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:07.210262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f63eb9-6caf-4f23-9b61-acb169ca7145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:07.210544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f63eb9-6caf-4f23-9b61-acb169ca7145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:07.220080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f63eb9-6caf-4f23-9b61-acb169ca7145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:07.221510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f63eb9-6caf-4f23-9b61-acb169ca7145 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:07.230719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 711/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:09.612293 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350d3ef-c0f4-4299-ae8f-282c5bbb80d1 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:09.612794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350d3ef-c0f4-4299-ae8f-282c5bbb80d1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:09.612918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350d3ef-c0f4-4299-ae8f-282c5bbb80d1 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:09.614241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 736/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:09.619461 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455b7dc-44f7-4c78-8a4c-c20ee411e416 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:09.619866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455b7dc-44f7-4c78-8a4c-c20ee411e416 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:09.620020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455b7dc-44f7-4c78-8a4c-c20ee411e416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:09.637988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9455b7dc-44f7-4c78-8a4c-c20ee411e416 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:09.639734 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9455b7dc-44f7-4c78-8a4c-c20ee411e416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:09.647637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 712/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:09.685635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350d3ef-c0f4-4299-ae8f-282c5bbb80d1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:09.696398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350d3ef-c0f4-4299-ae8f-282c5bbb80d1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:09.700151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910eabc-f27b-4f61-bd0c-758d4de051e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:09.700817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910eabc-f27b-4f61-bd0c-758d4de051e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:09.701103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910eabc-f27b-4f61-bd0c-758d4de051e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:09.701869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910eabc-f27b-4f61-bd0c-758d4de051e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:09.710982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910eabc-f27b-4f61-bd0c-758d4de051e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:09.712437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910eabc-f27b-4f61-bd0c-758d4de051e0 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:09.722311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 737/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:11.855514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef2ae1-f89a-4b81-8cb6-65dc5e19dd8d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:11.856053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef2ae1-f89a-4b81-8cb6-65dc5e19dd8d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:11.856053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef2ae1-f89a-4b81-8cb6-65dc5e19dd8d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:11.857333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 713/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:11.862906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a75ce-9c51-4900-966b-ce9f17b8e9de None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:11.863359 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a75ce-9c51-4900-966b-ce9f17b8e9de None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:11.863508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a75ce-9c51-4900-966b-ce9f17b8e9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:11.881605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-178a75ce-9c51-4900-966b-ce9f17b8e9de None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:11.882818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-178a75ce-9c51-4900-966b-ce9f17b8e9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:11.890696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 738/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:11.928288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef2ae1-f89a-4b81-8cb6-65dc5e19dd8d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:11.939398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef2ae1-f89a-4b81-8cb6-65dc5e19dd8d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:11.942825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ccf51-bacc-4a96-a487-68fda49cd0d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:11.943499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ccf51-bacc-4a96-a487-68fda49cd0d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:11.943786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ccf51-bacc-4a96-a487-68fda49cd0d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:11.944063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ccf51-bacc-4a96-a487-68fda49cd0d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:11.953337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ccf51-bacc-4a96-a487-68fda49cd0d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:11.954789 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ccf51-bacc-4a96-a487-68fda49cd0d6 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:11.964262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 714/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:14.082669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e74593-ab5d-480f-a631-0a9554a60c02 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:14.083260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e74593-ab5d-480f-a631-0a9554a60c02 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:14.083374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e74593-ab5d-480f-a631-0a9554a60c02 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:14.084691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 739/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:14.089961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd14334-7567-4733-8246-0b99fc8b2c08 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:14.090390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd14334-7567-4733-8246-0b99fc8b2c08 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:14.090526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd14334-7567-4733-8246-0b99fc8b2c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:14.108406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-bfd14334-7567-4733-8246-0b99fc8b2c08 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:14.109616 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bfd14334-7567-4733-8246-0b99fc8b2c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:14.117399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 715/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:14.154359 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e74593-ab5d-480f-a631-0a9554a60c02 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:14.165742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e74593-ab5d-480f-a631-0a9554a60c02 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:14.169280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f8a35-dd87-4da6-82f6-d3fd7cbddecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:14.169918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8a35-dd87-4da6-82f6-d3fd7cbddecc service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:14.170231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8a35-dd87-4da6-82f6-d3fd7cbddecc service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:14.170521 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f8a35-dd87-4da6-82f6-d3fd7cbddecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:14.179310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f8a35-dd87-4da6-82f6-d3fd7cbddecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:14.180751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394f8a35-dd87-4da6-82f6-d3fd7cbddecc service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:14.190107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 740/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:16.261818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84bef0-a9e9-48b4-b3cc-ab3468a9e934 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:16.262367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84bef0-a9e9-48b4-b3cc-ab3468a9e934 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:16.262367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84bef0-a9e9-48b4-b3cc-ab3468a9e934 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:16.264346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 716/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:16.269743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc95be-362b-472d-a103-520599bcac5d None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:16.270138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc95be-362b-472d-a103-520599bcac5d None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:16.270307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc95be-362b-472d-a103-520599bcac5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:16.288792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4ebc95be-362b-472d-a103-520599bcac5d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:16.289999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4ebc95be-362b-472d-a103-520599bcac5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:16.297845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 741/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:16.335944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84bef0-a9e9-48b4-b3cc-ab3468a9e934 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:16.347216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84bef0-a9e9-48b4-b3cc-ab3468a9e934 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:16.350756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cb9b9-3e01-4e84-bd9a-fb93d5592b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:16.351439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb9b9-3e01-4e84-bd9a-fb93d5592b99 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:16.351726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb9b9-3e01-4e84-bd9a-fb93d5592b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:16.352006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb9b9-3e01-4e84-bd9a-fb93d5592b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:16.361727 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cb9b9-3e01-4e84-bd9a-fb93d5592b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:16.363218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cb9b9-3e01-4e84-bd9a-fb93d5592b99 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:16.373231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 717/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:18.433639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa5511-8dd1-4d08-94a6-1309887f74f9 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:18.434185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa5511-8dd1-4d08-94a6-1309887f74f9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:18.434185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fa5511-8dd1-4d08-94a6-1309887f74f9 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:18.435626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 742/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:18.441410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290bd94-b9b4-489e-a5e2-a4d085d97329 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:18.441849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290bd94-b9b4-489e-a5e2-a4d085d97329 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:18.441959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290bd94-b9b4-489e-a5e2-a4d085d97329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:18.459820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0290bd94-b9b4-489e-a5e2-a4d085d97329 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:18.460936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0290bd94-b9b4-489e-a5e2-a4d085d97329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:18.468878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 718/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:18.505916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa5511-8dd1-4d08-94a6-1309887f74f9 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:18.516608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fa5511-8dd1-4d08-94a6-1309887f74f9 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:18.520634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641eb00b-1f36-4f55-b3c6-828d7f168abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:18.521377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eb00b-1f36-4f55-b3c6-828d7f168abd service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:18.521753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eb00b-1f36-4f55-b3c6-828d7f168abd service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:18.522121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641eb00b-1f36-4f55-b3c6-828d7f168abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:18.531352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641eb00b-1f36-4f55-b3c6-828d7f168abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:18.532776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641eb00b-1f36-4f55-b3c6-828d7f168abd service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:18.542634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 743/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:20.571872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c46686-46c3-4a1e-bbdc-aeea74febe15 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:20.572402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c46686-46c3-4a1e-bbdc-aeea74febe15 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:20.572402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c46686-46c3-4a1e-bbdc-aeea74febe15 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:20.573745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 719/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:20.579929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d1865-9137-4e82-ab23-6ce80942d126 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:20.580378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d1865-9137-4e82-ab23-6ce80942d126 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:20.580568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d1865-9137-4e82-ab23-6ce80942d126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:20.586464 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=79 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:07:20.598885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-659d1865-9137-4e82-ab23-6ce80942d126 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:20.600098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-659d1865-9137-4e82-ab23-6ce80942d126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:20.608220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 744/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:07:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:20.645057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c46686-46c3-4a1e-bbdc-aeea74febe15 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:20.650327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=73 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:07:20.656319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c46686-46c3-4a1e-bbdc-aeea74febe15 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:20.659831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d425a76-b2c2-44db-8953-9a8ec23bd433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:20.660465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d425a76-b2c2-44db-8953-9a8ec23bd433 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:20.660751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d425a76-b2c2-44db-8953-9a8ec23bd433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:20.661027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d425a76-b2c2-44db-8953-9a8ec23bd433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:20.669808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d425a76-b2c2-44db-8953-9a8ec23bd433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:20.671800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d425a76-b2c2-44db-8953-9a8ec23bd433 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:20.681057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 720/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:22.701239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe5b99-4d70-478b-bd22-8059d962d883 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:22.702398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe5b99-4d70-478b-bd22-8059d962d883 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:22.702398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afe5b99-4d70-478b-bd22-8059d962d883 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:22.703807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 745/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:07:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:22.709193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc277c-3f87-4c70-bbe0-7ea630c1929f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:22.709624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc277c-3f87-4c70-bbe0-7ea630c1929f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:22.709749 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc277c-3f87-4c70-bbe0-7ea630c1929f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:22.727361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f6dc277c-3f87-4c70-bbe0-7ea630c1929f None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:22.728555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f6dc277c-3f87-4c70-bbe0-7ea630c1929f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:22.736440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 721/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:22.772548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe5b99-4d70-478b-bd22-8059d962d883 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:22.783290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afe5b99-4d70-478b-bd22-8059d962d883 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:22.786827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61686049-f270-4e10-9afd-3ce741aa2c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:22.787497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61686049-f270-4e10-9afd-3ce741aa2c5d service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:22.787787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61686049-f270-4e10-9afd-3ce741aa2c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:22.788069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61686049-f270-4e10-9afd-3ce741aa2c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:22.796839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61686049-f270-4e10-9afd-3ce741aa2c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:22.798273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61686049-f270-4e10-9afd-3ce741aa2c5d service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:22.807676 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 746/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:24.826262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981939b8-9d93-4dbe-86f0-e50dae3e2d4c None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:24.826820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981939b8-9d93-4dbe-86f0-e50dae3e2d4c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:24.826820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981939b8-9d93-4dbe-86f0-e50dae3e2d4c None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:24.828230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 722/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:24.833322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ea0d8-50b3-4b3f-a6bf-395d79827467 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:24.833735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ea0d8-50b3-4b3f-a6bf-395d79827467 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:24.833894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ea0d8-50b3-4b3f-a6bf-395d79827467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:24.852076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-647ea0d8-50b3-4b3f-a6bf-395d79827467 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:24.853282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-647ea0d8-50b3-4b3f-a6bf-395d79827467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:24.861557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 747/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:24.896309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981939b8-9d93-4dbe-86f0-e50dae3e2d4c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:24.907308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981939b8-9d93-4dbe-86f0-e50dae3e2d4c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:24.910721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2387ea-811f-415c-9af5-af4b1d871c15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:24.911382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2387ea-811f-415c-9af5-af4b1d871c15 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:24.911665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2387ea-811f-415c-9af5-af4b1d871c15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:24.911941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2387ea-811f-415c-9af5-af4b1d871c15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:24.921111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2387ea-811f-415c-9af5-af4b1d871c15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:24.922536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2387ea-811f-415c-9af5-af4b1d871c15 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:24.931650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 723/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:34.846909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=11 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:07:34.927596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=14 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:07:44.460588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46493a-15ae-4146-b831-d85005190077 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:44.461131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46493a-15ae-4146-b831-d85005190077 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:44.461131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46493a-15ae-4146-b831-d85005190077 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:44.462473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 748/1471] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:44.467710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4c9b9-9935-40a0-a621-7a498c108cde None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:44.468160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4c9b9-9935-40a0-a621-7a498c108cde None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:44.468267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4c9b9-9935-40a0-a621-7a498c108cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:44.485547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-80d4c9b9-9935-40a0-a621-7a498c108cde None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:44.486773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-80d4c9b9-9935-40a0-a621-7a498c108cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:44.494520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 724/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:44.578095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85872ab-9626-408b-9f03-c2acdb863ba2 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:44.578389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85872ab-9626-408b-9f03-c2acdb863ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:44.578672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85872ab-9626-408b-9f03-c2acdb863ba2 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:44.580075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 749/1473] 10.222.0.23 () {54 vars in 962 bytes} [Tue Jul 5 15:07:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:44.585103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080c664-caa3-444b-9ef0-e888a53c7e23 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:44.585398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080c664-caa3-444b-9ef0-e888a53c7e23 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:44.585682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080c664-caa3-444b-9ef0-e888a53c7e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:44.603431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4080c664-caa3-444b-9ef0-e888a53c7e23 None None] MFA Rules not processed for user `450bcd0f22b840c299d08ce9433d6c85`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:44.604640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4080c664-caa3-444b-9ef0-e888a53c7e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:44.611509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 725/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Jul 5 15:07:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:44.618601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85872ab-9626-408b-9f03-c2acdb863ba2 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:44.629888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85872ab-9626-408b-9f03-c2acdb863ba2 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:44.633510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c8822-8ba1-4d69-afa7-e539bbf040de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450bcd0f22b840c299d08ce9433d6c85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:44.634135 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c8822-8ba1-4d69-afa7-e539bbf040de service glance] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:44.634423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c8822-8ba1-4d69-afa7-e539bbf040de service glance] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:44.634706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c8822-8ba1-4d69-afa7-e539bbf040de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:44.643636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c8822-8ba1-4d69-afa7-e539bbf040de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:44.645074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505c8822-8ba1-4d69-afa7-e539bbf040de service glance] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:44.654331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 750/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:07:46.803062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39260a-0583-432d-b954-b5b40f03df11 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:46.803630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39260a-0583-432d-b954-b5b40f03df11 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:46.803630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d39260a-0583-432d-b954-b5b40f03df11 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:46.804977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 726/1476] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:46.810064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dcbcc-1a3c-4374-90dd-8a507f2f05ae None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:46.810473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dcbcc-1a3c-4374-90dd-8a507f2f05ae None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:46.810633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73dcbcc-1a3c-4374-90dd-8a507f2f05ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:46.830584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-f73dcbcc-1a3c-4374-90dd-8a507f2f05ae None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:46.831840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f73dcbcc-1a3c-4374-90dd-8a507f2f05ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:46.839685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 751/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:46.869544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39260a-0583-432d-b954-b5b40f03df11 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:46.882853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d39260a-0583-432d-b954-b5b40f03df11 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:46.887264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fc834-c463-44f4-931a-ebd3d8b9db5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:46.887907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fc834-c463-44f4-931a-ebd3d8b9db5d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:46.888190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fc834-c463-44f4-931a-ebd3d8b9db5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:46.888490 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fc834-c463-44f4-931a-ebd3d8b9db5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:46.889252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fc834-c463-44f4-931a-ebd3d8b9db5d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:46.891186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-ee3fc834-c463-44f4-931a-ebd3d8b9db5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:07:46.891186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jul 05 15:07:46.891186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: Jul 05 15:07:46.892362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3fc834-c463-44f4-931a-ebd3d8b9db5d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:46.912245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 727/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 5 15:07:46 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:07:48.972906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830594d9-4deb-448e-8a28-c8b1f00a63f1 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:48.973465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830594d9-4deb-448e-8a28-c8b1f00a63f1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:48.973465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830594d9-4deb-448e-8a28-c8b1f00a63f1 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:48.974824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 752/1479] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:48.980016 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e872cd-a582-47aa-8d59-792605520f42 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:48.980459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e872cd-a582-47aa-8d59-792605520f42 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:48.980620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e872cd-a582-47aa-8d59-792605520f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:48.999018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-64e872cd-a582-47aa-8d59-792605520f42 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:49.000264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-64e872cd-a582-47aa-8d59-792605520f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:49.008103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 728/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:07:49.050505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830594d9-4deb-448e-8a28-c8b1f00a63f1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:49.064546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830594d9-4deb-448e-8a28-c8b1f00a63f1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:49.069870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552ebfb-2e7e-4fb1-922f-a951e9c4cde9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:49.070507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552ebfb-2e7e-4fb1-922f-a951e9c4cde9 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:49.070796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552ebfb-2e7e-4fb1-922f-a951e9c4cde9 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:49.071102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552ebfb-2e7e-4fb1-922f-a951e9c4cde9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:49.079334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552ebfb-2e7e-4fb1-922f-a951e9c4cde9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:49.080916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552ebfb-2e7e-4fb1-922f-a951e9c4cde9 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:49.090072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 753/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:51.279954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be8dde-2fe5-4f1c-a99b-9b5d10a9683b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:51.280493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be8dde-2fe5-4f1c-a99b-9b5d10a9683b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:51.280603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be8dde-2fe5-4f1c-a99b-9b5d10a9683b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:51.281934 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 729/1482] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:51.287132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c73c0-c42d-45f5-94cd-50b3b6841292 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:51.287574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c73c0-c42d-45f5-94cd-50b3b6841292 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:51.287680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c73c0-c42d-45f5-94cd-50b3b6841292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:51.304817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-2e5c73c0-c42d-45f5-94cd-50b3b6841292 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:51.305997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2e5c73c0-c42d-45f5-94cd-50b3b6841292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:51.314135 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 754/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:07:51.350522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be8dde-2fe5-4f1c-a99b-9b5d10a9683b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:51.361471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96be8dde-2fe5-4f1c-a99b-9b5d10a9683b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:51.364917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e884db-cfb7-49a1-9dc5-c955b0b0449a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:51.365551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e884db-cfb7-49a1-9dc5-c955b0b0449a service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:51.365839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e884db-cfb7-49a1-9dc5-c955b0b0449a service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:51.366120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e884db-cfb7-49a1-9dc5-c955b0b0449a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:51.375487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e884db-cfb7-49a1-9dc5-c955b0b0449a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:51.376918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e884db-cfb7-49a1-9dc5-c955b0b0449a service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:51.386154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 730/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:53.473951 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf24f47-8b99-4173-9bac-1ac25dea8a10 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:53.474480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf24f47-8b99-4173-9bac-1ac25dea8a10 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:53.474480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf24f47-8b99-4173-9bac-1ac25dea8a10 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:53.475841 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 755/1485] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:53.480890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a67d65-8f08-4a0c-be57-2eb5aa8e0376 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:53.481266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a67d65-8f08-4a0c-be57-2eb5aa8e0376 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:53.481454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a67d65-8f08-4a0c-be57-2eb5aa8e0376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:53.498737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-02a67d65-8f08-4a0c-be57-2eb5aa8e0376 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:53.500478 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-02a67d65-8f08-4a0c-be57-2eb5aa8e0376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:53.508161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 731/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:53.544963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf24f47-8b99-4173-9bac-1ac25dea8a10 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:53.555653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf24f47-8b99-4173-9bac-1ac25dea8a10 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:53.559163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c7ee6-54e7-4720-ae74-9baf63b3f405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:53.559793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c7ee6-54e7-4720-ae74-9baf63b3f405 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:53.560081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c7ee6-54e7-4720-ae74-9baf63b3f405 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:53.560360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c7ee6-54e7-4720-ae74-9baf63b3f405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:53.569451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c7ee6-54e7-4720-ae74-9baf63b3f405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:53.570883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c7ee6-54e7-4720-ae74-9baf63b3f405 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:53.580924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 756/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:55.769024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e08c99-ea47-4141-a346-bc222e25339e None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:55.769578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e08c99-ea47-4141-a346-bc222e25339e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:55.769578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e08c99-ea47-4141-a346-bc222e25339e None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:55.770960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 732/1488] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:07:55.776151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24387-a471-4fa0-8009-6d257524d6eb None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:55.776561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24387-a471-4fa0-8009-6d257524d6eb None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:55.776713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f24387-a471-4fa0-8009-6d257524d6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:55.794321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d0f24387-a471-4fa0-8009-6d257524d6eb None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:55.795535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d0f24387-a471-4fa0-8009-6d257524d6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:55.804146 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 757/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:55.843442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e08c99-ea47-4141-a346-bc222e25339e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:55.853950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e08c99-ea47-4141-a346-bc222e25339e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:55.857490 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179c92c-3444-4071-8795-c76cea56e062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:55.858168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179c92c-3444-4071-8795-c76cea56e062 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:55.859179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179c92c-3444-4071-8795-c76cea56e062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:55.859469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179c92c-3444-4071-8795-c76cea56e062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:55.868371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179c92c-3444-4071-8795-c76cea56e062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:55.869955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179c92c-3444-4071-8795-c76cea56e062 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:55.879531 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 733/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:07:58.073906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4ee5c-13c7-46ff-99e8-5f99b9430adb None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:58.074466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4ee5c-13c7-46ff-99e8-5f99b9430adb None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:58.074466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d4ee5c-13c7-46ff-99e8-5f99b9430adb None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:58.075869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 758/1491] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 5 15:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:07:58.081432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93dd33-1680-4c09-a844-71fa98266d79 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:58.081878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93dd33-1680-4c09-a844-71fa98266d79 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:58.081985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93dd33-1680-4c09-a844-71fa98266d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:58.099703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4d93dd33-1680-4c09-a844-71fa98266d79 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:07:58.100897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4d93dd33-1680-4c09-a844-71fa98266d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:07:58.109266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 734/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 5 15:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:07:58.147580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4ee5c-13c7-46ff-99e8-5f99b9430adb None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:07:58.158779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d4ee5c-13c7-46ff-99e8-5f99b9430adb None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:07:58.162260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d68f9-49eb-48fd-bdd9-a40d0c28fdba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:07:58.162895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d68f9-49eb-48fd-bdd9-a40d0c28fdba service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:07:58.163221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d68f9-49eb-48fd-bdd9-a40d0c28fdba service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:07:58.163503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d68f9-49eb-48fd-bdd9-a40d0c28fdba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:07:58.172188 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d68f9-49eb-48fd-bdd9-a40d0c28fdba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:07:58.173614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d68f9-49eb-48fd-bdd9-a40d0c28fdba service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:07:58.182824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 759/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:07:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:08:08.094819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=33 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:08:08.178725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=33 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:09:29.514429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec3c81-c40a-4153-8718-ec858ddf1883 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:29.514997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec3c81-c40a-4153-8718-ec858ddf1883 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:29.514997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec3c81-c40a-4153-8718-ec858ddf1883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:29.533119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ddec3c81-c40a-4153-8718-ec858ddf1883 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:09:29.534334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ddec3c81-c40a-4153-8718-ec858ddf1883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:09:29.542120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 735/1494] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:09:29.550470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641e29b-416f-454c-bd4c-4a54931fd6af None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:29.550979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641e29b-416f-454c-bd4c-4a54931fd6af None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:29.551117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641e29b-416f-454c-bd4c-4a54931fd6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:29.568940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c641e29b-416f-454c-bd4c-4a54931fd6af None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:09:29.570121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c641e29b-416f-454c-bd4c-4a54931fd6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:09:29.577801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 760/1495] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:09:29.582323 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec3c81-c40a-4153-8718-ec858ddf1883 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:09:29.591799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec3c81-c40a-4153-8718-ec858ddf1883 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:09:29.596756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91734381-8708-4b35-81a3-b86b8953dfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:09:29.597404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91734381-8708-4b35-81a3-b86b8953dfdf admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:29.597690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91734381-8708-4b35-81a3-b86b8953dfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:29.597964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91734381-8708-4b35-81a3-b86b8953dfdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:29.598693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91734381-8708-4b35-81a3-b86b8953dfdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:09:29.599011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91734381-8708-4b35-81a3-b86b8953dfdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:09:29.600054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-91734381-8708-4b35-81a3-b86b8953dfdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:09:29.600054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jul 05 15:09:29.600054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: Jul 05 15:09:29.600949 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91734381-8708-4b35-81a3-b86b8953dfdf admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:09:29.612769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 736/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:09:29.620684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641e29b-416f-454c-bd4c-4a54931fd6af None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:09:29.629587 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641e29b-416f-454c-bd4c-4a54931fd6af None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:09:29.634499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f52d2-63ce-4181-96ba-8262f6e5a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:09:29.634696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f52d2-63ce-4181-96ba-8262f6e5a84f admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:29.634978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f52d2-63ce-4181-96ba-8262f6e5a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:29.635284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f52d2-63ce-4181-96ba-8262f6e5a84f admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:29.635973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f52d2-63ce-4181-96ba-8262f6e5a84f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:09:29.637729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: WARNING py.warnings [None req-201f52d2-63ce-4181-96ba-8262f6e5a84f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:09:29.637729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: warnings.warn(msg) Jul 05 15:09:29.637729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: Jul 05 15:09:29.638907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201f52d2-63ce-4181-96ba-8262f6e5a84f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:09:29.674631 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 761/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:09:29 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:09:29.678814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91734381-8708-4b35-81a3-b86b8953dfdf admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:09:29.688525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91734381-8708-4b35-81a3-b86b8953dfdf admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:09:29.745038 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226f540-04f7-4d14-9eea-03bf3421b16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:09:29.745697 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226f540-04f7-4d14-9eea-03bf3421b16e admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:29.745982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226f540-04f7-4d14-9eea-03bf3421b16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:29.746262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226f540-04f7-4d14-9eea-03bf3421b16e admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:29.746958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226f540-04f7-4d14-9eea-03bf3421b16e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:09:29.748011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-1226f540-04f7-4d14-9eea-03bf3421b16e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:09:29.748011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jul 05 15:09:29.748011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: Jul 05 15:09:29.749219 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226f540-04f7-4d14-9eea-03bf3421b16e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:09:29.768779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 737/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:09:29 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:09:29.772871 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f52d2-63ce-4181-96ba-8262f6e5a84f admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:09:29.782978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201f52d2-63ce-4181-96ba-8262f6e5a84f admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:09:29.787772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b27633-a1bf-4b2f-8278-cd635dd20fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:09:29.788430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b27633-a1bf-4b2f-8278-cd635dd20fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:29.788717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b27633-a1bf-4b2f-8278-cd635dd20fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:29.788997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b27633-a1bf-4b2f-8278-cd635dd20fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:29.789720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b27633-a1bf-4b2f-8278-cd635dd20fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:09:29.790725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: WARNING py.warnings [None req-e8b27633-a1bf-4b2f-8278-cd635dd20fdb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:09:29.790725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: warnings.warn(msg) Jul 05 15:09:29.790725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: Jul 05 15:09:29.791684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b27633-a1bf-4b2f-8278-cd635dd20fdb admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:09:29.804693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 762/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:09:29.808732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226f540-04f7-4d14-9eea-03bf3421b16e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:09:29.819185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226f540-04f7-4d14-9eea-03bf3421b16e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:09:29.823711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33694fc-7025-4f7e-8b99-77d26c907600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:09:29.824363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33694fc-7025-4f7e-8b99-77d26c907600 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:29.824650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33694fc-7025-4f7e-8b99-77d26c907600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:29.824930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33694fc-7025-4f7e-8b99-77d26c907600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:29.825652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33694fc-7025-4f7e-8b99-77d26c907600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:09:29.826682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-f33694fc-7025-4f7e-8b99-77d26c907600 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:09:29.826682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jul 05 15:09:29.826682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: Jul 05 15:09:29.827605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33694fc-7025-4f7e-8b99-77d26c907600 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:09:29.840225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 738/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:09:29.844065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b27633-a1bf-4b2f-8278-cd635dd20fdb admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:09:29.853586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b27633-a1bf-4b2f-8278-cd635dd20fdb admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:09:29.857975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aec9d1-4cca-4df8-b04c-df7cd8da5d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:09:29.858846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aec9d1-4cca-4df8-b04c-df7cd8da5d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:29.859651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aec9d1-4cca-4df8-b04c-df7cd8da5d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:29.859990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aec9d1-4cca-4df8-b04c-df7cd8da5d5e admin admin] PATH_INFO: `/v3/projects/da72601a80c540d2adaf09c78a6644ab/users/e5976fe8d9db4824a68deeb5357d8550/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:29.873860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aec9d1-4cca-4df8-b04c-df7cd8da5d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=da72601a80c540d2adaf09c78a6644ab, user_id=e5976fe8d9db4824a68deeb5357d8550, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:09:29.875485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: WARNING py.warnings [None req-c0aec9d1-4cca-4df8-b04c-df7cd8da5d5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:09:29.875485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: warnings.warn(msg) Jul 05 15:09:29.875485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: Jul 05 15:09:29.877401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aec9d1-4cca-4df8-b04c-df7cd8da5d5e admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:09:29.889580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 763/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:09:29 2022] PUT /identity/v3/projects/da72601a80c540d2adaf09c78a6644ab/users/e5976fe8d9db4824a68deeb5357d8550/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:09:29.901561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2e376-30b1-401c-9a76-f696510dc42f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:29.901857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2e376-30b1-401c-9a76-f696510dc42f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:29.902147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2e376-30b1-401c-9a76-f696510dc42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:29.928276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9fd2e376-30b1-401c-9a76-f696510dc42f None None] MFA Rules not processed for user `e5976fe8d9db4824a68deeb5357d8550`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:09:29.929493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9fd2e376-30b1-401c-9a76-f696510dc42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:09:29.992118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 739/1502] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:09:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:09:30.005392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd0a67-1008-48ec-bc58-fa8e6c251760 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:30.005687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd0a67-1008-48ec-bc58-fa8e6c251760 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:30.005968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fd0a67-1008-48ec-bc58-fa8e6c251760 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:30.007395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 764/1503] 10.222.0.23 () {54 vars in 963 bytes} [Tue Jul 5 15:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:09:30.011994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3984-9aed-49d4-acbb-9f97169046f3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:30.012289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3984-9aed-49d4-acbb-9f97169046f3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:30.012570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c3984-9aed-49d4-acbb-9f97169046f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:30.029973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-011c3984-9aed-49d4-acbb-9f97169046f3 None None] MFA Rules not processed for user `8ad7cee9ffed415b9a30176b1888e562`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:09:30.031656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-011c3984-9aed-49d4-acbb-9f97169046f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:09:30.073229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:09:30.081634 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 740/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Tue Jul 5 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:09:30.089199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd0a67-1008-48ec-bc58-fa8e6c251760 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:09:30.100719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fd0a67-1008-48ec-bc58-fa8e6c251760 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:09:30.104217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c5326-3ea4-45de-8889-bcf9fc70c877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:09:30.104850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c5326-3ea4-45de-8889-bcf9fc70c877 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:30.105137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c5326-3ea4-45de-8889-bcf9fc70c877 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:30.105417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c5326-3ea4-45de-8889-bcf9fc70c877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:30.114118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c5326-3ea4-45de-8889-bcf9fc70c877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:09:30.115593 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c5326-3ea4-45de-8889-bcf9fc70c877 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:09:30.124154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 765/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:09:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:09:30.165091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55142ede-8d52-46d5-a228-286c1accc436 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:30.165380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55142ede-8d52-46d5-a228-286c1accc436 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:30.165660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55142ede-8d52-46d5-a228-286c1accc436 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:30.166930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 741/1506] 10.222.0.23 () {52 vars in 829 bytes} [Tue Jul 5 15:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:09:30.170476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c5326-3ea4-45de-8889-bcf9fc70c877 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:09:30.231843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c5326-3ea4-45de-8889-bcf9fc70c877 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:09:30.236353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:09:30.237218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:30.237515 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:30.237794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] PATH_INFO: `/v3/users/e5976fe8d9db4824a68deeb5357d8550` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:30.238968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5976fe8d9db4824a68deeb5357d8550)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:09:30.240015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: WARNING py.warnings [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:09:30.240015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: warnings.warn(msg) Jul 05 15:09:30.240015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: Jul 05 15:09:30.241214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:09:30.302689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5976fe8d9db4824a68deeb5357d8550'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:09:30.303305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5976fe8d9db4824a68deeb5357d8550'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:09:30.304017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5976fe8d9db4824a68deeb5357d8550'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:09:30.316569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5976fe8d9db4824a68deeb5357d8550'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:09:30.325227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5976fe8d9db4824a68deeb5357d8550'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:09:30.342057 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:09:30.351391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 766/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:09:30 2022] DELETE /identity/v3/users/e5976fe8d9db4824a68deeb5357d8550 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:09:30.355768 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e4408-fa44-4d87-84bf-6b61724c4a30 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:30.356053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e4408-fa44-4d87-84bf-6b61724c4a30 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:30.356330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4e4408-fa44-4d87-84bf-6b61724c4a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:30.373666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ea4e4408-fa44-4d87-84bf-6b61724c4a30 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:09:30.375382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ea4e4408-fa44-4d87-84bf-6b61724c4a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:09:30.431944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 742/1508] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:09:30.441842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:09:30.443300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:09:30.498422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc7ecb-da71-45da-abc4-a61be6977d87 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:09:30.502592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814ce60-c84e-4771-82bd-3fcd154513a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:09:30.503261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814ce60-c84e-4771-82bd-3fcd154513a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:30.503555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814ce60-c84e-4771-82bd-3fcd154513a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:30.503840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814ce60-c84e-4771-82bd-3fcd154513a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:30.512628 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814ce60-c84e-4771-82bd-3fcd154513a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:09:30.514064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2814ce60-c84e-4771-82bd-3fcd154513a8 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:09:30.523356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 767/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:09:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:09:30.848353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e4408-fa44-4d87-84bf-6b61724c4a30 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:09:30.849827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ea4e4408-fa44-4d87-84bf-6b61724c4a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:09:30.862532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4e4408-fa44-4d87-84bf-6b61724c4a30 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:09:30.867647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:09:30.868560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:09:30.868850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:09:30.869128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] PATH_INFO: `/v3/projects/da72601a80c540d2adaf09c78a6644ab` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:09:30.870426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da72601a80c540d2adaf09c78a6644ab)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:09:30.871480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:09:30.871480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jul 05 15:09:30.871480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: Jul 05 15:09:30.872688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:09:30.933155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da72601a80c540d2adaf09c78a6644ab', 'request_id': 'req-54cb3027-cc93-4f21-b5da-7e7b996bc299'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:09:30.942838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da72601a80c540d2adaf09c78a6644ab', 'request_id': 'req-54cb3027-cc93-4f21-b5da-7e7b996bc299'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:09:30.956444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da72601a80c540d2adaf09c78a6644ab', 'request_id': 'req-54cb3027-cc93-4f21-b5da-7e7b996bc299'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:09:30.968056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] The token cache is being invalidate because project da72601a80c540d2adaf09c78a6644ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:09:30.968373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:09:30.969557 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-54cb3027-cc93-4f21-b5da-7e7b996bc299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:09:30.971298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 743/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:09:30 2022] DELETE /identity/v3/projects/da72601a80c540d2adaf09c78a6644ab => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:09:40.519344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:09:40.965013 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:11:12.340970 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb551ad-394f-4374-b067-931c84c5ba86 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:12.341531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb551ad-394f-4374-b067-931c84c5ba86 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:12.341531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb551ad-394f-4374-b067-931c84c5ba86 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:12.342856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 768/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:11:12.349996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ec087-7a25-4434-bdf9-f175b4c12df3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:12.349996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ec087-7a25-4434-bdf9-f175b4c12df3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:12.349996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ec087-7a25-4434-bdf9-f175b4c12df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:12.370381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-289ec087-7a25-4434-bdf9-f175b4c12df3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:11:12.371630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-289ec087-7a25-4434-bdf9-f175b4c12df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:12.429865 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 744/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:11:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:11:12.584805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb551ad-394f-4374-b067-931c84c5ba86 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:11:12.586503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-aeb551ad-394f-4374-b067-931c84c5ba86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:12.663535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb551ad-394f-4374-b067-931c84c5ba86 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:11:12.668896 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ae51d-b03c-40d1-b3f1-00f13675284f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:11:12.669540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ae51d-b03c-40d1-b3f1-00f13675284f service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:12.669829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ae51d-b03c-40d1-b3f1-00f13675284f service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:12.670110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ae51d-b03c-40d1-b3f1-00f13675284f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:12.678609 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ae51d-b03c-40d1-b3f1-00f13675284f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:11:12.680235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ae51d-b03c-40d1-b3f1-00f13675284f service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:11:12.689468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 769/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:11:13.509252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55bb34-6016-4efb-851b-f7b73fd439fd None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:13.509719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55bb34-6016-4efb-851b-f7b73fd439fd None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:13.509719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55bb34-6016-4efb-851b-f7b73fd439fd None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:13.511021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 745/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:11:13.516014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5d1b9-4be8-413a-8a41-f0bea8dd2951 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:13.516470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5d1b9-4be8-413a-8a41-f0bea8dd2951 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:13.516584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5d1b9-4be8-413a-8a41-f0bea8dd2951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:13.534907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c0b5d1b9-4be8-413a-8a41-f0bea8dd2951 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:11:13.536795 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c0b5d1b9-4be8-413a-8a41-f0bea8dd2951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:13.544576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 770/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:11:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:11:13.694476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55bb34-6016-4efb-851b-f7b73fd439fd None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:11:13.710817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55bb34-6016-4efb-851b-f7b73fd439fd None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:11:13.716362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e079d-bf76-41e9-b177-10d550c8c5c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:11:13.717030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e079d-bf76-41e9-b177-10d550c8c5c0 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:13.717314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e079d-bf76-41e9-b177-10d550c8c5c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:13.717616 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e079d-bf76-41e9-b177-10d550c8c5c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:13.727032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e079d-bf76-41e9-b177-10d550c8c5c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:11:13.728666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e079d-bf76-41e9-b177-10d550c8c5c0 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:11:13.738244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 746/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:11:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:11:13.795671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c726-f98d-48dc-80e2-74d038340710 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:13.796072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c726-f98d-48dc-80e2-74d038340710 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:13.796418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c726-f98d-48dc-80e2-74d038340710 None None] PATH_INFO: `/v3` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:13.797911 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 771/1517] 10.222.0.33 () {54 vars in 925 bytes} [Tue Jul 5 15:11:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:11:13.804286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087dc8c-d3ef-4c9a-9fea-5c45140932e9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:13.804582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087dc8c-d3ef-4c9a-9fea-5c45140932e9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:13.804863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087dc8c-d3ef-4c9a-9fea-5c45140932e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:13.826002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c087dc8c-d3ef-4c9a-9fea-5c45140932e9 None None] MFA Rules not processed for user `c68f1297771e4c03a897dd1f0ac21cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:11:13.827257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c087dc8c-d3ef-4c9a-9fea-5c45140932e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:13.889502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 747/1518] 10.222.0.33 () {58 vars in 1015 bytes} [Tue Jul 5 15:11:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:11:22.848080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a4e77-2171-43f3-b576-689ad6d36041 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:22.848585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a4e77-2171-43f3-b576-689ad6d36041 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:22.848585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a4e77-2171-43f3-b576-689ad6d36041 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:22.849988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 772/1519] 10.222.0.23 () {54 vars in 960 bytes} [Tue Jul 5 15:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:11:22.854885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6035-1b05-40e9-ae57-b749127d142f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:22.855295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6035-1b05-40e9-ae57-b749127d142f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:22.855485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6035-1b05-40e9-ae57-b749127d142f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:22.873448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-703f6035-1b05-40e9-ae57-b749127d142f None None] MFA Rules not processed for user `c68f1297771e4c03a897dd1f0ac21cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:11:22.874625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-703f6035-1b05-40e9-ae57-b749127d142f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:22.882900 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 748/1520] 10.222.0.23 () {58 vars in 1057 bytes} [Tue Jul 5 15:11:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:11:22.890145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a4e77-2171-43f3-b576-689ad6d36041 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:11:22.903276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a4e77-2171-43f3-b576-689ad6d36041 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:11:22.908081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0803b55-4332-4823-80a3-a86ec81276ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:11:22.908701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0803b55-4332-4823-80a3-a86ec81276ac service placement] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:22.908985 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0803b55-4332-4823-80a3-a86ec81276ac service placement] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:22.909265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0803b55-4332-4823-80a3-a86ec81276ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:22.917793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0803b55-4332-4823-80a3-a86ec81276ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:11:22.919423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0803b55-4332-4823-80a3-a86ec81276ac service placement] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:11:22.929566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 773/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:11:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:11:24.954154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf54c4-6325-4275-b292-6e4051dd991d None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:24.954725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf54c4-6325-4275-b292-6e4051dd991d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:24.954725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf54c4-6325-4275-b292-6e4051dd991d None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:24.956130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 749/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:11:24.961779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f4964-dedc-4b81-bfe3-40411d484ebd None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:24.962201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f4964-dedc-4b81-bfe3-40411d484ebd None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:24.962347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f4964-dedc-4b81-bfe3-40411d484ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:24.983176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-929f4964-dedc-4b81-bfe3-40411d484ebd None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:11:24.984407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-929f4964-dedc-4b81-bfe3-40411d484ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:24.992411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 774/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:11:25.144112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf54c4-6325-4275-b292-6e4051dd991d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:11:25.158984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf54c4-6325-4275-b292-6e4051dd991d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:11:25.164387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399de015-1363-4d4b-826b-54b855e11ec7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:11:25.165023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399de015-1363-4d4b-826b-54b855e11ec7 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:25.165310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399de015-1363-4d4b-826b-54b855e11ec7 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:25.165592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399de015-1363-4d4b-826b-54b855e11ec7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:25.173541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399de015-1363-4d4b-826b-54b855e11ec7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:11:25.175181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399de015-1363-4d4b-826b-54b855e11ec7 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:11:25.184523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 750/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:11:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:11:25.995823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6639-9178-440d-aec6-97cc7df895b8 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:25.996349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6639-9178-440d-aec6-97cc7df895b8 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:25.996349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40db6639-9178-440d-aec6-97cc7df895b8 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:25.997723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 775/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 5 15:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:11:26.002451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a8149-ce81-4bc9-a3ab-5bab01556eba None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:26.002878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a8149-ce81-4bc9-a3ab-5bab01556eba None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:26.003017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a8149-ce81-4bc9-a3ab-5bab01556eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:26.020574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1c4a8149-ce81-4bc9-a3ab-5bab01556eba None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:11:26.021764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1c4a8149-ce81-4bc9-a3ab-5bab01556eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:26.029419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 751/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 5 15:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:11:26.178862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db6639-9178-440d-aec6-97cc7df895b8 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:11:26.192913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40db6639-9178-440d-aec6-97cc7df895b8 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:11:26.198282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460009c-92c7-40e7-a90e-82eb9db8073a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:11:26.198927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460009c-92c7-40e7-a90e-82eb9db8073a service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:26.199244 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460009c-92c7-40e7-a90e-82eb9db8073a service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:26.199527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460009c-92c7-40e7-a90e-82eb9db8073a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:26.208432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460009c-92c7-40e7-a90e-82eb9db8073a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:11:26.210290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460009c-92c7-40e7-a90e-82eb9db8073a service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:11:26.219636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 776/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:11:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:11:30.650505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671532e-3510-4844-b363-675e095aa375 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:30.651076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671532e-3510-4844-b363-675e095aa375 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:30.651076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671532e-3510-4844-b363-675e095aa375 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:30.652491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 752/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:11:30.657518 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ad1c7-4abf-4315-a167-8205e12990d7 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:30.657913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ad1c7-4abf-4315-a167-8205e12990d7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:30.658089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ad1c7-4abf-4315-a167-8205e12990d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:30.675508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-457ad1c7-4abf-4315-a167-8205e12990d7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:11:30.677215 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-457ad1c7-4abf-4315-a167-8205e12990d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:30.684882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 777/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:11:30.757085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671532e-3510-4844-b363-675e095aa375 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:11:30.758759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f671532e-3510-4844-b363-675e095aa375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:30.816047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671532e-3510-4844-b363-675e095aa375 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:11:30.820132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d4fbb-bf4d-4680-94e5-f44295b55be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:11:30.820808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d4fbb-bf4d-4680-94e5-f44295b55be8 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:30.821092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d4fbb-bf4d-4680-94e5-f44295b55be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:30.821370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d4fbb-bf4d-4680-94e5-f44295b55be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:30.830252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d4fbb-bf4d-4680-94e5-f44295b55be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:11:30.831769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d4fbb-bf4d-4680-94e5-f44295b55be8 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:11:30.837860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:11:30.843167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 753/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:11:32.830303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c977e72-100f-4ca2-837c-e4128e55a5f6 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:32.830866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c977e72-100f-4ca2-837c-e4128e55a5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:32.833016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c977e72-100f-4ca2-837c-e4128e55a5f6 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:32.835160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 778/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:11:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:11:32.840500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1a468-6726-4b8c-996f-e92bfeb23f0e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:32.840945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1a468-6726-4b8c-996f-e92bfeb23f0e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:32.841053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1a468-6726-4b8c-996f-e92bfeb23f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:32.858480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-aeb1a468-6726-4b8c-996f-e92bfeb23f0e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:11:32.859693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-aeb1a468-6726-4b8c-996f-e92bfeb23f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:32.867315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 754/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:11:32.902520 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c977e72-100f-4ca2-837c-e4128e55a5f6 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:11:32.913151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c977e72-100f-4ca2-837c-e4128e55a5f6 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:11:32.916666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5677ce3-ca52-4f59-a664-e39622c5f8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:11:32.917302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5677ce3-ca52-4f59-a664-e39622c5f8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:32.917594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5677ce3-ca52-4f59-a664-e39622c5f8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:32.917878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5677ce3-ca52-4f59-a664-e39622c5f8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:32.927504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5677ce3-ca52-4f59-a664-e39622c5f8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:11:32.928940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5677ce3-ca52-4f59-a664-e39622c5f8d0 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:11:32.938174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 779/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:11:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:11:36.784485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c4938-1974-455a-8d5e-b73cfc532a57 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:36.785591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c4938-1974-455a-8d5e-b73cfc532a57 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:36.785707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c4938-1974-455a-8d5e-b73cfc532a57 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:36.786997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 755/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:11:36.791968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39bd27-d4b8-4c86-8700-14b1950e5fdc None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:36.792372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39bd27-d4b8-4c86-8700-14b1950e5fdc None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:36.792560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39bd27-d4b8-4c86-8700-14b1950e5fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:36.810010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4e39bd27-d4b8-4c86-8700-14b1950e5fdc None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:11:36.811328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4e39bd27-d4b8-4c86-8700-14b1950e5fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:36.819008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 780/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:11:37.117317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c4938-1974-455a-8d5e-b73cfc532a57 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:11:37.119013 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-245c4938-1974-455a-8d5e-b73cfc532a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:37.174261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c4938-1974-455a-8d5e-b73cfc532a57 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:11:37.177713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5c9de9-9f6f-453d-b880-d00f7a04b93e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:11:37.178345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c9de9-9f6f-453d-b880-d00f7a04b93e service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:37.178627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c9de9-9f6f-453d-b880-d00f7a04b93e service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:37.178901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5c9de9-9f6f-453d-b880-d00f7a04b93e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:37.187505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c9de9-9f6f-453d-b880-d00f7a04b93e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:11:37.188921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5c9de9-9f6f-453d-b880-d00f7a04b93e service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:11:37.198736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 756/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:11:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:11:39.255572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ea7da-fe17-48e5-b720-c27d5a037c40 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:39.256156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ea7da-fe17-48e5-b720-c27d5a037c40 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:39.256156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ea7da-fe17-48e5-b720-c27d5a037c40 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:39.257519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 781/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 5 15:11:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:11:39.262725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e782e-e529-4b4a-a0c4-0491786199c6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:39.263200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e782e-e529-4b4a-a0c4-0491786199c6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:39.263315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e782e-e529-4b4a-a0c4-0491786199c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:39.281801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e01e782e-e529-4b4a-a0c4-0491786199c6 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:11:39.283022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e01e782e-e529-4b4a-a0c4-0491786199c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:39.290759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 757/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 5 15:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:11:39.588616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ea7da-fe17-48e5-b720-c27d5a037c40 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:11:39.590272 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c63ea7da-fe17-48e5-b720-c27d5a037c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:11:39.602971 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ea7da-fe17-48e5-b720-c27d5a037c40 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:11:39.606401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547735d-55ad-43d8-b870-11660a503eff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:11:39.607035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547735d-55ad-43d8-b870-11660a503eff service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:11:39.607394 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547735d-55ad-43d8-b870-11660a503eff service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:11:39.607657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547735d-55ad-43d8-b870-11660a503eff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:11:39.616352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547735d-55ad-43d8-b870-11660a503eff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:11:39.617775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547735d-55ad-43d8-b870-11660a503eff service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:11:39.627156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 782/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:11:49.276769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=32 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:11:49.623027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=70 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:11.756296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80334f-6ef8-4331-a1b4-19c1556f9866 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:11.756848 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80334f-6ef8-4331-a1b4-19c1556f9866 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:11.756942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80334f-6ef8-4331-a1b4-19c1556f9866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:11.774054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af665020-5939-4ba9-9ed7-0853e02b88f5 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:11.774645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af665020-5939-4ba9-9ed7-0853e02b88f5 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:11.774645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af665020-5939-4ba9-9ed7-0853e02b88f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:11.793562 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9f80334f-6ef8-4331-a1b4-19c1556f9866 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:11.794807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9f80334f-6ef8-4331-a1b4-19c1556f9866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:11.798369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-af665020-5939-4ba9-9ed7-0853e02b88f5 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:11.803851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-af665020-5939-4ba9-9ed7-0853e02b88f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:11.813271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 758/1540] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:13:11.820731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 783/1541] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:11.823866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3416fdd-64ee-4295-ab1c-a408806fd573 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:11.824150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3416fdd-64ee-4295-ab1c-a408806fd573 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:11.824438 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3416fdd-64ee-4295-ab1c-a408806fd573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:11.829579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e417879-c6b0-4814-bfe0-afafaf50f618 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:11.829851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e417879-c6b0-4814-bfe0-afafaf50f618 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:11.830099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e417879-c6b0-4814-bfe0-afafaf50f618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:11.849934 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d3416fdd-64ee-4295-ab1c-a408806fd573 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:11.851485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d3416fdd-64ee-4295-ab1c-a408806fd573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:11.861320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5e417879-c6b0-4814-bfe0-afafaf50f618 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:11.862532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5e417879-c6b0-4814-bfe0-afafaf50f618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:11.864709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 759/1542] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:11.868923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3416fdd-64ee-4295-ab1c-a408806fd573 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:11.879074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3416fdd-64ee-4295-ab1c-a408806fd573 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:11.881767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 784/1543] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:13:11.886353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e417879-c6b0-4814-bfe0-afafaf50f618 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:11.896505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb594ad-ba27-4896-bc4e-8b8beed19e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:11.897203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb594ad-ba27-4896-bc4e-8b8beed19e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:11.897495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb594ad-ba27-4896-bc4e-8b8beed19e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:11.897779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb594ad-ba27-4896-bc4e-8b8beed19e10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:11.898522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb594ad-ba27-4896-bc4e-8b8beed19e10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:11.898836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb594ad-ba27-4896-bc4e-8b8beed19e10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:11.900504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb594ad-ba27-4896-bc4e-8b8beed19e10 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:11.900907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e417879-c6b0-4814-bfe0-afafaf50f618 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:11.910485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b1d01-d54d-4ca1-8867-eb19d4a03ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:11.911259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b1d01-d54d-4ca1-8867-eb19d4a03ae9 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:11.911554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b1d01-d54d-4ca1-8867-eb19d4a03ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:11.911960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b1d01-d54d-4ca1-8867-eb19d4a03ae9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:11.912706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b1d01-d54d-4ca1-8867-eb19d4a03ae9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:11.913019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b1d01-d54d-4ca1-8867-eb19d4a03ae9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:11.914049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: WARNING py.warnings [None req-023b1d01-d54d-4ca1-8867-eb19d4a03ae9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:13:11.914049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: warnings.warn(msg) Jul 05 15:13:11.914049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: Jul 05 15:13:11.914940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b1d01-d54d-4ca1-8867-eb19d4a03ae9 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:11.918198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 760/1544] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:13:11.927764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb594ad-ba27-4896-bc4e-8b8beed19e10 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:11.942415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb594ad-ba27-4896-bc4e-8b8beed19e10 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:11.944154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 785/1545] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:11.952041 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b1d01-d54d-4ca1-8867-eb19d4a03ae9 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:11.955743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a761d2-b282-42d3-939a-130d1d9b852b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:11.956440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a761d2-b282-42d3-939a-130d1d9b852b admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:11.956727 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a761d2-b282-42d3-939a-130d1d9b852b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:11.957008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a761d2-b282-42d3-939a-130d1d9b852b admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:11.958257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a761d2-b282-42d3-939a-130d1d9b852b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:11.962305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-18a761d2-b282-42d3-939a-130d1d9b852b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:13:11.962305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jul 05 15:13:11.962305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: Jul 05 15:13:11.962305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a761d2-b282-42d3-939a-130d1d9b852b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:11.974226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b1d01-d54d-4ca1-8867-eb19d4a03ae9 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:11.983946 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc1662-1600-4e9a-9c08-1de37dcaff10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:11.984633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc1662-1600-4e9a-9c08-1de37dcaff10 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:11.984923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc1662-1600-4e9a-9c08-1de37dcaff10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:11.985206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc1662-1600-4e9a-9c08-1de37dcaff10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:11.985934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc1662-1600-4e9a-9c08-1de37dcaff10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:11.988222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc1662-1600-4e9a-9c08-1de37dcaff10 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.021678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 761/1546] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:11 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:12.022953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1b900-a00a-4da9-ad14-90ace88bcb2d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.023282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1b900-a00a-4da9-ad14-90ace88bcb2d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.023636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1b900-a00a-4da9-ad14-90ace88bcb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.047131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 786/1547] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:11 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:13:12.048190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc1662-1600-4e9a-9c08-1de37dcaff10 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.072585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f8c1b900-a00a-4da9-ad14-90ace88bcb2d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.074312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f8c1b900-a00a-4da9-ad14-90ace88bcb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.178639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc1662-1600-4e9a-9c08-1de37dcaff10 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.184892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197cfd0-6eb0-46b4-93a7-6ea5567c10c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.185579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197cfd0-6eb0-46b4-93a7-6ea5567c10c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.189844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197cfd0-6eb0-46b4-93a7-6ea5567c10c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.189844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197cfd0-6eb0-46b4-93a7-6ea5567c10c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.189844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197cfd0-6eb0-46b4-93a7-6ea5567c10c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.189844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: WARNING py.warnings [None req-f197cfd0-6eb0-46b4-93a7-6ea5567c10c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:13:12.189844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: warnings.warn(msg) Jul 05 15:13:12.189844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: Jul 05 15:13:12.189844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f197cfd0-6eb0-46b4-93a7-6ea5567c10c0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.197964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 762/1548] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.204862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e999f-5cee-4bd4-9b2f-59e4f2701ea2 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.205057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e999f-5cee-4bd4-9b2f-59e4f2701ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.205643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e999f-5cee-4bd4-9b2f-59e4f2701ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.214964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 787/1549] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:12.215635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197cfd0-6eb0-46b4-93a7-6ea5567c10c0 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.226335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197cfd0-6eb0-46b4-93a7-6ea5567c10c0 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.231089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b8e398-b289-4cb8-8b49-fdaf329b1ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.231785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8e398-b289-4cb8-8b49-fdaf329b1ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.232071 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8e398-b289-4cb8-8b49-fdaf329b1ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.232347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b8e398-b289-4cb8-8b49-fdaf329b1ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.233050 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8e398-b289-4cb8-8b49-fdaf329b1ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.235254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b8e398-b289-4cb8-8b49-fdaf329b1ab2 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.258219 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 788/1550] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:12.260011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0fb2-4da1-4fe0-ab87-6e1d86886bf5 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.260309 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0fb2-4da1-4fe0-ab87-6e1d86886bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.260584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de0fb2-4da1-4fe0-ab87-6e1d86886bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.280576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-49de0fb2-4da1-4fe0-ab87-6e1d86886bf5 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.281742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-49de0fb2-4da1-4fe0-ab87-6e1d86886bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.290467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 789/1551] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.291818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f28729-55b2-4d69-86db-14ec246272c0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.292111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f28729-55b2-4d69-86db-14ec246272c0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.292390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f28729-55b2-4d69-86db-14ec246272c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.311908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-50f28729-55b2-4d69-86db-14ec246272c0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.313166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-50f28729-55b2-4d69-86db-14ec246272c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.321817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 790/1552] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.323328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaa285-fc33-4864-90ec-6f28635e0643 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.323629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaa285-fc33-4864-90ec-6f28635e0643 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.323909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eaa285-fc33-4864-90ec-6f28635e0643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.342206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-12eaa285-fc33-4864-90ec-6f28635e0643 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.343437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-12eaa285-fc33-4864-90ec-6f28635e0643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.344993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c01e999f-5cee-4bd4-9b2f-59e4f2701ea2 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.346184 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c01e999f-5cee-4bd4-9b2f-59e4f2701ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.351895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 791/1553] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.353175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022605b4-85d3-494e-98f4-82bc59de57c4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.353465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022605b4-85d3-494e-98f4-82bc59de57c4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.353743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022605b4-85d3-494e-98f4-82bc59de57c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.361584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 763/1554] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.362897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889271c-aad9-4698-97bb-112feb2fe1db None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.363273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889271c-aad9-4698-97bb-112feb2fe1db None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.363955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889271c-aad9-4698-97bb-112feb2fe1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.376733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-022605b4-85d3-494e-98f4-82bc59de57c4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.377922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-022605b4-85d3-494e-98f4-82bc59de57c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.383841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-6889271c-aad9-4698-97bb-112feb2fe1db None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.384989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6889271c-aad9-4698-97bb-112feb2fe1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.385850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 792/1555] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.386679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022605b4-85d3-494e-98f4-82bc59de57c4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.395106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 764/1556] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.395992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022605b4-85d3-494e-98f4-82bc59de57c4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.399952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889271c-aad9-4698-97bb-112feb2fe1db None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.400661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f6140-4d57-4ebb-89fe-a689e430ea6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.401311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f6140-4d57-4ebb-89fe-a689e430ea6d admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.401665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f6140-4d57-4ebb-89fe-a689e430ea6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.401953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f6140-4d57-4ebb-89fe-a689e430ea6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.402673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f6140-4d57-4ebb-89fe-a689e430ea6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.404320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f6140-4d57-4ebb-89fe-a689e430ea6d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.410048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889271c-aad9-4698-97bb-112feb2fe1db None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.417728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 793/1557] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:12.417880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea7822-1b3f-4d58-8368-20f382edfb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.417880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea7822-1b3f-4d58-8368-20f382edfb5a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.417880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea7822-1b3f-4d58-8368-20f382edfb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.418137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36dad-7806-4bdf-8fa6-65b1dabef67d None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.418224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea7822-1b3f-4d58-8368-20f382edfb5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.418316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36dad-7806-4bdf-8fa6-65b1dabef67d None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.418316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c36dad-7806-4bdf-8fa6-65b1dabef67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.418468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea7822-1b3f-4d58-8368-20f382edfb5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.418797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea7822-1b3f-4d58-8368-20f382edfb5a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.429164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 765/1558] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:12.430142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f485c9-c71f-4524-9bb4-b04583311d05 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.430429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f485c9-c71f-4524-9bb4-b04583311d05 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.430704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f485c9-c71f-4524-9bb4-b04583311d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.434424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-59c36dad-7806-4bdf-8fa6-65b1dabef67d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.435664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-59c36dad-7806-4bdf-8fa6-65b1dabef67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.444262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 794/1559] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.445542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb4800-48dd-4ab0-96dc-38766225ac78 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.445839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb4800-48dd-4ab0-96dc-38766225ac78 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.446122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb4800-48dd-4ab0-96dc-38766225ac78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.448582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f4f485c9-c71f-4524-9bb4-b04583311d05 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.449770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f4f485c9-c71f-4524-9bb4-b04583311d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.457548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 766/1560] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.458834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9e87c-0f5a-451a-b61f-79cb9210f05b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.459216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9e87c-0f5a-451a-b61f-79cb9210f05b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.459501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9e87c-0f5a-451a-b61f-79cb9210f05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.467883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-62cb4800-48dd-4ab0-96dc-38766225ac78 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.469072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-62cb4800-48dd-4ab0-96dc-38766225ac78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.477777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-16f9e87c-0f5a-451a-b61f-79cb9210f05b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.477997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 795/1561] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.479248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-16f9e87c-0f5a-451a-b61f-79cb9210f05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.481922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb4800-48dd-4ab0-96dc-38766225ac78 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.486842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 767/1562] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.488069 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28a9d7-2b56-426b-83c5-e4d5cc80ee3a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.488356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28a9d7-2b56-426b-83c5-e4d5cc80ee3a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.488630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce28a9d7-2b56-426b-83c5-e4d5cc80ee3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.494722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb4800-48dd-4ab0-96dc-38766225ac78 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.499524 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c25d4d-bfcc-494c-b61d-a3757d2cf0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.500202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c25d4d-bfcc-494c-b61d-a3757d2cf0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.500486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c25d4d-bfcc-494c-b61d-a3757d2cf0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.500759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c25d4d-bfcc-494c-b61d-a3757d2cf0c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.501473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c25d4d-bfcc-494c-b61d-a3757d2cf0c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:12.501782 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c25d4d-bfcc-494c-b61d-a3757d2cf0c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.503505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c25d4d-bfcc-494c-b61d-a3757d2cf0c9 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.507340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ce28a9d7-2b56-426b-83c5-e4d5cc80ee3a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:12.508564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ce28a9d7-2b56-426b-83c5-e4d5cc80ee3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:12.515138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 796/1563] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:12.516096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c25d4d-bfcc-494c-b61d-a3757d2cf0c9 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.519157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 768/1564] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:12.520247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28a9d7-2b56-426b-83c5-e4d5cc80ee3a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.525814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c25d4d-bfcc-494c-b61d-a3757d2cf0c9 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.530331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71998c-31b5-4f7c-8f15-76e519d695d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.530476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce28a9d7-2b56-426b-83c5-e4d5cc80ee3a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.531072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71998c-31b5-4f7c-8f15-76e519d695d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.531303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71998c-31b5-4f7c-8f15-76e519d695d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.531584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71998c-31b5-4f7c-8f15-76e519d695d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.532302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71998c-31b5-4f7c-8f15-76e519d695d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.533907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71998c-31b5-4f7c-8f15-76e519d695d1 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.535005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe8e67-ac07-41a9-87c3-123778a6bea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.535680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe8e67-ac07-41a9-87c3-123778a6bea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.536465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe8e67-ac07-41a9-87c3-123778a6bea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.536736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe8e67-ac07-41a9-87c3-123778a6bea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.537448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe8e67-ac07-41a9-87c3-123778a6bea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.539122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe8e67-ac07-41a9-87c3-123778a6bea7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.544604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 797/1565] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:12.545386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71998c-31b5-4f7c-8f15-76e519d695d1 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.552861 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 769/1566] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:12.552861 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe8e67-ac07-41a9-87c3-123778a6bea7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.555126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71998c-31b5-4f7c-8f15-76e519d695d1 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.559929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68d1a3-f771-4ee0-93e5-6f53bdb7da18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.560572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68d1a3-f771-4ee0-93e5-6f53bdb7da18 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.560859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68d1a3-f771-4ee0-93e5-6f53bdb7da18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.561139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68d1a3-f771-4ee0-93e5-6f53bdb7da18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.561883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68d1a3-f771-4ee0-93e5-6f53bdb7da18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:12.562178 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68d1a3-f771-4ee0-93e5-6f53bdb7da18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.562321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe8e67-ac07-41a9-87c3-123778a6bea7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.563811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a68d1a3-f771-4ee0-93e5-6f53bdb7da18 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.566671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:12.566935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44426571-4154-434f-9128-90d3205adccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.567617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44426571-4154-434f-9128-90d3205adccc admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.567907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44426571-4154-434f-9128-90d3205adccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.568186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44426571-4154-434f-9128-90d3205adccc admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.568915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44426571-4154-434f-9128-90d3205adccc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:12.569221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44426571-4154-434f-9128-90d3205adccc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.570918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44426571-4154-434f-9128-90d3205adccc admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.579214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 798/1567] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:12.580263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68d1a3-f771-4ee0-93e5-6f53bdb7da18 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.584118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 770/1568] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:12.589694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44426571-4154-434f-9128-90d3205adccc admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.590451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68d1a3-f771-4ee0-93e5-6f53bdb7da18 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.595154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d20ec7-2d7e-4e39-9a94-42e744d9bc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.595802 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d20ec7-2d7e-4e39-9a94-42e744d9bc81 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.596090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d20ec7-2d7e-4e39-9a94-42e744d9bc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.596373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d20ec7-2d7e-4e39-9a94-42e744d9bc81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.597101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d20ec7-2d7e-4e39-9a94-42e744d9bc81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:12.597408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d20ec7-2d7e-4e39-9a94-42e744d9bc81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.599607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d20ec7-2d7e-4e39-9a94-42e744d9bc81 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.600793 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44426571-4154-434f-9128-90d3205adccc admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.605651 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b50b3-97d5-4777-a18b-8683f367fa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.606293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b50b3-97d5-4777-a18b-8683f367fa45 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.606580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b50b3-97d5-4777-a18b-8683f367fa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.606860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b50b3-97d5-4777-a18b-8683f367fa45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.607620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b50b3-97d5-4777-a18b-8683f367fa45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:12.607931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b50b3-97d5-4777-a18b-8683f367fa45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.609559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b50b3-97d5-4777-a18b-8683f367fa45 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.611772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 799/1569] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:12.612548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d20ec7-2d7e-4e39-9a94-42e744d9bc81 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.621941 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d20ec7-2d7e-4e39-9a94-42e744d9bc81 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.630895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 771/1570] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:12.630895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b50b3-97d5-4777-a18b-8683f367fa45 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.631176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c767646-78d6-4a5b-b891-b707d8dc73c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.631176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767646-78d6-4a5b-b891-b707d8dc73c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.631176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767646-78d6-4a5b-b891-b707d8dc73c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.631176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c767646-78d6-4a5b-b891-b707d8dc73c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.631751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c767646-78d6-4a5b-b891-b707d8dc73c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:12.632095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c767646-78d6-4a5b-b891-b707d8dc73c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.633713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c767646-78d6-4a5b-b891-b707d8dc73c5 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.634812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b50b3-97d5-4777-a18b-8683f367fa45 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.639407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab165910-9190-414a-bafd-48f3433eb2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.640027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab165910-9190-414a-bafd-48f3433eb2a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.640315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab165910-9190-414a-bafd-48f3433eb2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.640588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab165910-9190-414a-bafd-48f3433eb2a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.641258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab165910-9190-414a-bafd-48f3433eb2a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.643132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab165910-9190-414a-bafd-48f3433eb2a0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.652448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 800/1571] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:12.652448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c767646-78d6-4a5b-b891-b707d8dc73c5 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.661764 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c767646-78d6-4a5b-b891-b707d8dc73c5 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.666475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5a760-cc53-4c5a-b0a1-9143bbf64135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.667455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5a760-cc53-4c5a-b0a1-9143bbf64135 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.667754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5a760-cc53-4c5a-b0a1-9143bbf64135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.668037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5a760-cc53-4c5a-b0a1-9143bbf64135 admin admin] PATH_INFO: `/v3/projects/542b790e5cc5402fb8a1438496a7fc5a/users/0036ce88dc5247bc83f35de8bf8e68b1/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.670382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 772/1572] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:12.671375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab165910-9190-414a-bafd-48f3433eb2a0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.681455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:12.686017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5a760-cc53-4c5a-b0a1-9143bbf64135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=542b790e5cc5402fb8a1438496a7fc5a, user_id=0036ce88dc5247bc83f35de8bf8e68b1, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.688726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e5a760-cc53-4c5a-b0a1-9143bbf64135 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.702437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 801/1573] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:12 2022] PUT /identity/v3/projects/542b790e5cc5402fb8a1438496a7fc5a/users/0036ce88dc5247bc83f35de8bf8e68b1/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:12.703377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5a760-cc53-4c5a-b0a1-9143bbf64135 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.749661 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab165910-9190-414a-bafd-48f3433eb2a0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.763729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e5a760-cc53-4c5a-b0a1-9143bbf64135 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.768436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb9b06-5551-434c-81b5-941d5900206a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.769101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb9b06-5551-434c-81b5-941d5900206a admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.769391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb9b06-5551-434c-81b5-941d5900206a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.769667 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb9b06-5551-434c-81b5-941d5900206a admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.770863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb9b06-5551-434c-81b5-941d5900206a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.772855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb9b06-5551-434c-81b5-941d5900206a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.798455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 802/1574] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:12.799310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb9b06-5551-434c-81b5-941d5900206a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.810596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a785-43bb-4a2b-a9cf-da9be1fb0965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.811520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a785-43bb-4a2b-a9cf-da9be1fb0965 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.811804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a785-43bb-4a2b-a9cf-da9be1fb0965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.812086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a785-43bb-4a2b-a9cf-da9be1fb0965 admin admin] PATH_INFO: `/v3/projects/772abc72002b4256b0f600577232889a/users/ef098bbb9ee94bf4aadb4b59516890fc/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.826249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8a785-43bb-4a2b-a9cf-da9be1fb0965 admin admin] RBAC: Authorizing `identity:create_grant(project_id=772abc72002b4256b0f600577232889a, user_id=ef098bbb9ee94bf4aadb4b59516890fc, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.827327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-adb8a785-43bb-4a2b-a9cf-da9be1fb0965 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:13:12.827327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jul 05 15:13:12.827327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: Jul 05 15:13:12.829276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8a785-43bb-4a2b-a9cf-da9be1fb0965 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.843231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 773/1575] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:12 2022] PUT /identity/v3/projects/772abc72002b4256b0f600577232889a/users/ef098bbb9ee94bf4aadb4b59516890fc/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:12.844018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a785-43bb-4a2b-a9cf-da9be1fb0965 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.861856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb9b06-5551-434c-81b5-941d5900206a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.895618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:12.906025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a785-43bb-4a2b-a9cf-da9be1fb0965 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:12.911242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60673758-c46c-4c67-9546-78127dc86cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.911904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60673758-c46c-4c67-9546-78127dc86cad admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.912192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60673758-c46c-4c67-9546-78127dc86cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.912471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60673758-c46c-4c67-9546-78127dc86cad admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.913165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60673758-c46c-4c67-9546-78127dc86cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.915178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60673758-c46c-4c67-9546-78127dc86cad admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.919774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41537955-d911-422a-822b-0fdcd013b5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:12.920442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41537955-d911-422a-822b-0fdcd013b5bc admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:12.920733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41537955-d911-422a-822b-0fdcd013b5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:12.921015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41537955-d911-422a-822b-0fdcd013b5bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:12.921708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41537955-d911-422a-822b-0fdcd013b5bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:12.923677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41537955-d911-422a-822b-0fdcd013b5bc admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:12.945074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 774/1576] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:12.946120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60673758-c46c-4c67-9546-78127dc86cad admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.949887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 803/1577] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/projects => generated 367 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:12.950750 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41537955-d911-422a-822b-0fdcd013b5bc admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:12.975006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:13.012066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60673758-c46c-4c67-9546-78127dc86cad admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.013192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41537955-d911-422a-822b-0fdcd013b5bc admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.016591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc06a87-0c9d-40c4-af4a-afacd90d85e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.017247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc06a87-0c9d-40c4-af4a-afacd90d85e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.017531 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc06a87-0c9d-40c4-af4a-afacd90d85e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.017694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05d361-a50a-41c2-8510-a3a326b72493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.017874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc06a87-0c9d-40c4-af4a-afacd90d85e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.018345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05d361-a50a-41c2-8510-a3a326b72493 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.018490 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc06a87-0c9d-40c4-af4a-afacd90d85e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.018620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05d361-a50a-41c2-8510-a3a326b72493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.018896 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05d361-a50a-41c2-8510-a3a326b72493 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.019618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05d361-a50a-41c2-8510-a3a326b72493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.020492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc06a87-0c9d-40c4-af4a-afacd90d85e6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.021607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05d361-a50a-41c2-8510-a3a326b72493 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.045987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 775/1578] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:13.047232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 804/1579] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:12 2022] POST /identity/v3/projects => generated 405 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:13.048432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26739b2d-6227-44bf-8599-1993ac3e1d07 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.048780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26739b2d-6227-44bf-8599-1993ac3e1d07 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.049236 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26739b2d-6227-44bf-8599-1993ac3e1d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.049927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc06a87-0c9d-40c4-af4a-afacd90d85e6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.074572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-26739b2d-6227-44bf-8599-1993ac3e1d07 None None] MFA Rules not processed for user `0036ce88dc5247bc83f35de8bf8e68b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:13.076127 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-26739b2d-6227-44bf-8599-1993ac3e1d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:13.111436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc06a87-0c9d-40c4-af4a-afacd90d85e6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.140722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 805/1580] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:13.141747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26739b2d-6227-44bf-8599-1993ac3e1d07 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.174291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e0edc-6f76-45bf-a988-ccea71d6e4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.175133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0edc-6f76-45bf-a988-ccea71d6e4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.175389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0edc-6f76-45bf-a988-ccea71d6e4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.175665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e0edc-6f76-45bf-a988-ccea71d6e4c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.177083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e0edc-6f76-45bf-a988-ccea71d6e4c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.179076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4e0edc-6f76-45bf-a988-ccea71d6e4c8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.188440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:13.200342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 776/1581] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:13 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:13.204601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123ec75-a1f3-4393-8abe-84081022e6be None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.204896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123ec75-a1f3-4393-8abe-84081022e6be None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.205198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123ec75-a1f3-4393-8abe-84081022e6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.214335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26739b2d-6227-44bf-8599-1993ac3e1d07 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.218978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070254f7-3696-45bb-b5e1-7e2952ca7ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.219701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070254f7-3696-45bb-b5e1-7e2952ca7ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.219993 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070254f7-3696-45bb-b5e1-7e2952ca7ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.220283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070254f7-3696-45bb-b5e1-7e2952ca7ee2 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.220962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070254f7-3696-45bb-b5e1-7e2952ca7ee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.223068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070254f7-3696-45bb-b5e1-7e2952ca7ee2 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.224283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2123ec75-a1f3-4393-8abe-84081022e6be None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:13.225475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2123ec75-a1f3-4393-8abe-84081022e6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:13.232952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 777/1582] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:13.234482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123ec75-a1f3-4393-8abe-84081022e6be None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.244577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123ec75-a1f3-4393-8abe-84081022e6be None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.245525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 806/1583] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:13 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:13.246329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070254f7-3696-45bb-b5e1-7e2952ca7ee2 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.249342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e9a38-bfba-4aa8-b356-9605edcc468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.249998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e9a38-bfba-4aa8-b356-9605edcc468f admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.250279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e9a38-bfba-4aa8-b356-9605edcc468f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.250553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e9a38-bfba-4aa8-b356-9605edcc468f admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.251265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e9a38-bfba-4aa8-b356-9605edcc468f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.253198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e9a38-bfba-4aa8-b356-9605edcc468f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.255800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070254f7-3696-45bb-b5e1-7e2952ca7ee2 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.260425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4f162-8bbf-4d09-b7f9-a27f590de525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.261081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4f162-8bbf-4d09-b7f9-a27f590de525 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.261364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4f162-8bbf-4d09-b7f9-a27f590de525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.261645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4f162-8bbf-4d09-b7f9-a27f590de525 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.262332 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4f162-8bbf-4d09-b7f9-a27f590de525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.264502 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4f162-8bbf-4d09-b7f9-a27f590de525 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.272942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 778/1584] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:13 2022] POST /identity/v3/users => generated 289 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:13.273855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e9a38-bfba-4aa8-b356-9605edcc468f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.283731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e9a38-bfba-4aa8-b356-9605edcc468f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.286464 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 807/1585] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:13 2022] POST /identity/v3/users => generated 284 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:13.287286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4f162-8bbf-4d09-b7f9-a27f590de525 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.288937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be26b8-4e75-429f-9a30-7a7f0365e5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.289884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be26b8-4e75-429f-9a30-7a7f0365e5aa admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.290170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be26b8-4e75-429f-9a30-7a7f0365e5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.290444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be26b8-4e75-429f-9a30-7a7f0365e5aa admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.291160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be26b8-4e75-429f-9a30-7a7f0365e5aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.293079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be26b8-4e75-429f-9a30-7a7f0365e5aa admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.296596 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4f162-8bbf-4d09-b7f9-a27f590de525 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.301233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce87a64-4a65-4e68-b9b3-04ac2b78a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.302384 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce87a64-4a65-4e68-b9b3-04ac2b78a604 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.302671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce87a64-4a65-4e68-b9b3-04ac2b78a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.302957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce87a64-4a65-4e68-b9b3-04ac2b78a604 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.303671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce87a64-4a65-4e68-b9b3-04ac2b78a604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.305622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce87a64-4a65-4e68-b9b3-04ac2b78a604 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.314061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 779/1586] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:13 2022] POST /identity/v3/users => generated 295 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:13.315395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c75d5d-27ea-40be-8227-fd3742c8c97b None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.315797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c75d5d-27ea-40be-8227-fd3742c8c97b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.315956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c75d5d-27ea-40be-8227-fd3742c8c97b None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.317267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 780/1587] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jul 5 15:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:13:13.318064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c75d5d-27ea-40be-8227-fd3742c8c97b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.326742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:13.329142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 808/1588] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:13 2022] POST /identity/v3/users => generated 303 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:13.330014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce87a64-4a65-4e68-b9b3-04ac2b78a604 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.331606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8ce87a64-4a65-4e68-b9b3-04ac2b78a604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:13.336851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c75d5d-27ea-40be-8227-fd3742c8c97b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.341626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87039d-3d1f-4a2b-9675-49ff814c887c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.342289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87039d-3d1f-4a2b-9675-49ff814c887c admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.342609 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87039d-3d1f-4a2b-9675-49ff814c887c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.342891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87039d-3d1f-4a2b-9675-49ff814c887c admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.343677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87039d-3d1f-4a2b-9675-49ff814c887c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.345431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87039d-3d1f-4a2b-9675-49ff814c887c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.355836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 781/1589] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.356621 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87039d-3d1f-4a2b-9675-49ff814c887c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.366720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87039d-3d1f-4a2b-9675-49ff814c887c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.371293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106e29a-6e6e-49ce-8fb6-79baba1dded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.371929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106e29a-6e6e-49ce-8fb6-79baba1dded9 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.372215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106e29a-6e6e-49ce-8fb6-79baba1dded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.372496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106e29a-6e6e-49ce-8fb6-79baba1dded9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.373205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106e29a-6e6e-49ce-8fb6-79baba1dded9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.374795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106e29a-6e6e-49ce-8fb6-79baba1dded9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.385303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 782/1590] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.386087 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106e29a-6e6e-49ce-8fb6-79baba1dded9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.390753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce87a64-4a65-4e68-b9b3-04ac2b78a604 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.394434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dd7a7-6fee-453c-959b-74d8ee67d375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.395203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106e29a-6e6e-49ce-8fb6-79baba1dded9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.395323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dd7a7-6fee-453c-959b-74d8ee67d375 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.395526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dd7a7-6fee-453c-959b-74d8ee67d375 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.395813 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45dd7a7-6fee-453c-959b-74d8ee67d375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.399573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a905c25-cf59-47d8-91a9-93fc410bd5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.400189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a905c25-cf59-47d8-91a9-93fc410bd5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.400472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a905c25-cf59-47d8-91a9-93fc410bd5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.400745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a905c25-cf59-47d8-91a9-93fc410bd5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.401450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a905c25-cf59-47d8-91a9-93fc410bd5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.403343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a905c25-cf59-47d8-91a9-93fc410bd5b4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.404858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dd7a7-6fee-453c-959b-74d8ee67d375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.406299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45dd7a7-6fee-453c-959b-74d8ee67d375 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.413272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 783/1591] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.414356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a905c25-cf59-47d8-91a9-93fc410bd5b4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.416955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 809/1592] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:13.417765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dd7a7-6fee-453c-959b-74d8ee67d375 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.425039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a905c25-cf59-47d8-91a9-93fc410bd5b4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.427839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45dd7a7-6fee-453c-959b-74d8ee67d375 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.430435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371dfdc-33b5-4985-a92c-e6c3da0899d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.431122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371dfdc-33b5-4985-a92c-e6c3da0899d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.431534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371dfdc-33b5-4985-a92c-e6c3da0899d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.431817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371dfdc-33b5-4985-a92c-e6c3da0899d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.432572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371dfdc-33b5-4985-a92c-e6c3da0899d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.432954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07489a1c-2491-42f0-a5b3-3121c08066cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.433614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07489a1c-2491-42f0-a5b3-3121c08066cd admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.434250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07489a1c-2491-42f0-a5b3-3121c08066cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.434381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371dfdc-33b5-4985-a92c-e6c3da0899d9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.434490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07489a1c-2491-42f0-a5b3-3121c08066cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.436027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07489a1c-2491-42f0-a5b3-3121c08066cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.438264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07489a1c-2491-42f0-a5b3-3121c08066cd admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.450157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 810/1593] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.453039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4826f93-f2ec-4633-a182-36db12d18088 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.453039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4826f93-f2ec-4633-a182-36db12d18088 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.453039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4826f93-f2ec-4633-a182-36db12d18088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.453445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 784/1594] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.454471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371dfdc-33b5-4985-a92c-e6c3da0899d9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.464722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371dfdc-33b5-4985-a92c-e6c3da0899d9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.470072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb0412-d73c-44ae-99c0-c32da2ddf0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.470312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb0412-d73c-44ae-99c0-c32da2ddf0d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.470546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb0412-d73c-44ae-99c0-c32da2ddf0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.470829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb0412-d73c-44ae-99c0-c32da2ddf0d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.471587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb0412-d73c-44ae-99c0-c32da2ddf0d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.471935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d4826f93-f2ec-4633-a182-36db12d18088 None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:13.473514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d4826f93-f2ec-4633-a182-36db12d18088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:13.473691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb0412-d73c-44ae-99c0-c32da2ddf0d8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.486239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 785/1595] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.490653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb0412-d73c-44ae-99c0-c32da2ddf0d8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.501304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb0412-d73c-44ae-99c0-c32da2ddf0d8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.506881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038c14b-4685-4137-b285-faeb7463bb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.507534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038c14b-4685-4137-b285-faeb7463bb93 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.507823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038c14b-4685-4137-b285-faeb7463bb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.508106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038c14b-4685-4137-b285-faeb7463bb93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.508842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038c14b-4685-4137-b285-faeb7463bb93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.510461 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038c14b-4685-4137-b285-faeb7463bb93 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.522118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 786/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.522986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038c14b-4685-4137-b285-faeb7463bb93 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.533394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038c14b-4685-4137-b285-faeb7463bb93 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.538160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54734287-5ea9-4a13-aa5b-0a14d3a9a099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.538830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54734287-5ea9-4a13-aa5b-0a14d3a9a099 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.539167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54734287-5ea9-4a13-aa5b-0a14d3a9a099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.539445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54734287-5ea9-4a13-aa5b-0a14d3a9a099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.540177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54734287-5ea9-4a13-aa5b-0a14d3a9a099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.541793 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54734287-5ea9-4a13-aa5b-0a14d3a9a099 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.553640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 787/1597] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.554368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54734287-5ea9-4a13-aa5b-0a14d3a9a099 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.563721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 811/1598] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jul 5 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:13.564372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54734287-5ea9-4a13-aa5b-0a14d3a9a099 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.565187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4826f93-f2ec-4633-a182-36db12d18088 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.570085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e7048-0ddf-4fca-ac2e-af571014e818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.570762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e7048-0ddf-4fca-ac2e-af571014e818 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.571153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e7048-0ddf-4fca-ac2e-af571014e818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.571439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289e7048-0ddf-4fca-ac2e-af571014e818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.572177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e7048-0ddf-4fca-ac2e-af571014e818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.573915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289e7048-0ddf-4fca-ac2e-af571014e818 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.580537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4826f93-f2ec-4633-a182-36db12d18088 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.584810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 788/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.585726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e7048-0ddf-4fca-ac2e-af571014e818 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.586892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba5cad-dcc9-461a-90c9-12c55b6bb903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.587768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba5cad-dcc9-461a-90c9-12c55b6bb903 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.588438 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba5cad-dcc9-461a-90c9-12c55b6bb903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.588874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba5cad-dcc9-461a-90c9-12c55b6bb903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.589760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba5cad-dcc9-461a-90c9-12c55b6bb903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.591717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba5cad-dcc9-461a-90c9-12c55b6bb903 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.595609 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289e7048-0ddf-4fca-ac2e-af571014e818 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.600490 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5db9a-551c-4300-8f8a-8713dae5f00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.601162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5db9a-551c-4300-8f8a-8713dae5f00c admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.601449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5db9a-551c-4300-8f8a-8713dae5f00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.601757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee5db9a-551c-4300-8f8a-8713dae5f00c admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.602486 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5db9a-551c-4300-8f8a-8713dae5f00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.604121 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee5db9a-551c-4300-8f8a-8713dae5f00c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.604474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 812/1600] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.605576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba5cad-dcc9-461a-90c9-12c55b6bb903 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.612890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:13.615708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 789/1601] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.618330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba5cad-dcc9-461a-90c9-12c55b6bb903 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.619762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5db9a-551c-4300-8f8a-8713dae5f00c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.625451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe03d80-856c-43d5-895d-74d43528d522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.626605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe03d80-856c-43d5-895d-74d43528d522 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.626605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe03d80-856c-43d5-895d-74d43528d522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.626834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe03d80-856c-43d5-895d-74d43528d522 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.627572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe03d80-856c-43d5-895d-74d43528d522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.630544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe03d80-856c-43d5-895d-74d43528d522 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.630805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee5db9a-551c-4300-8f8a-8713dae5f00c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.636509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00365a88-b733-4423-a689-598cd1cbdbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.639763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00365a88-b733-4423-a689-598cd1cbdbc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.639763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00365a88-b733-4423-a689-598cd1cbdbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.639763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00365a88-b733-4423-a689-598cd1cbdbc8 admin admin] PATH_INFO: `/v3/projects/7ccd2398b3334bac9f0b3e188cb85324/users/177b36dc3e014cd388292a75e5424518/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.647826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 813/1602] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:13.649098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe03d80-856c-43d5-895d-74d43528d522 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.658835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00365a88-b733-4423-a689-598cd1cbdbc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ccd2398b3334bac9f0b3e188cb85324, user_id=177b36dc3e014cd388292a75e5424518, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.661763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00365a88-b733-4423-a689-598cd1cbdbc8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.663288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe03d80-856c-43d5-895d-74d43528d522 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.668599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40847c7-45a3-4527-8835-c1f4adb2b24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.669731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40847c7-45a3-4527-8835-c1f4adb2b24a admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.670007 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40847c7-45a3-4527-8835-c1f4adb2b24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.670414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40847c7-45a3-4527-8835-c1f4adb2b24a admin admin] PATH_INFO: `/v3/projects/b8d16bd4e4864d3089a95df440ca5894/users/0187789947064682a8d9e685c2aadb9e/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.677985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 790/1603] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:13 2022] PUT /identity/v3/projects/7ccd2398b3334bac9f0b3e188cb85324/users/177b36dc3e014cd388292a75e5424518/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:13.678832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00365a88-b733-4423-a689-598cd1cbdbc8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.697501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40847c7-45a3-4527-8835-c1f4adb2b24a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d16bd4e4864d3089a95df440ca5894, user_id=0187789947064682a8d9e685c2aadb9e, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.701416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40847c7-45a3-4527-8835-c1f4adb2b24a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.716841 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 814/1604] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:13 2022] PUT /identity/v3/projects/b8d16bd4e4864d3089a95df440ca5894/users/0187789947064682a8d9e685c2aadb9e/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:13.717151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40847c7-45a3-4527-8835-c1f4adb2b24a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.789826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00365a88-b733-4423-a689-598cd1cbdbc8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.822947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40847c7-45a3-4527-8835-c1f4adb2b24a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.827778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf909f-682d-4c11-b063-dc3647110e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.828682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf909f-682d-4c11-b063-dc3647110e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.829018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf909f-682d-4c11-b063-dc3647110e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.829308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf909f-682d-4c11-b063-dc3647110e36 admin admin] PATH_INFO: `/v3/projects/a0ccca42ac0a4113bba224c8d076dd81/users/ba0f3ef2b024464a9c3e71f5d639ca69/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.838181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:13.847680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf909f-682d-4c11-b063-dc3647110e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ccca42ac0a4113bba224c8d076dd81, user_id=ba0f3ef2b024464a9c3e71f5d639ca69, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.852523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf909f-682d-4c11-b063-dc3647110e36 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.873404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 815/1605] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:13 2022] PUT /identity/v3/projects/a0ccca42ac0a4113bba224c8d076dd81/users/ba0f3ef2b024464a9c3e71f5d639ca69/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:13.874597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf909f-682d-4c11-b063-dc3647110e36 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.878884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746844c4-14b2-4397-af76-5120e8cf1b6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.880488 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746844c4-14b2-4397-af76-5120e8cf1b6f service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.881003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746844c4-14b2-4397-af76-5120e8cf1b6f service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.882896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746844c4-14b2-4397-af76-5120e8cf1b6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.958443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf909f-682d-4c11-b063-dc3647110e36 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.964068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d806e-8610-47da-aa93-0330a3f23c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.964958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d806e-8610-47da-aa93-0330a3f23c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.965160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746844c4-14b2-4397-af76-5120e8cf1b6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.965285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d806e-8610-47da-aa93-0330a3f23c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.965505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d806e-8610-47da-aa93-0330a3f23c32 admin admin] PATH_INFO: `/v3/projects/6c0f74e2f5794c0189b647e29646eeb2/users/8bbc454737084513baa37072cd160e73/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:13.966950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746844c4-14b2-4397-af76-5120e8cf1b6f service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.978655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 791/1606] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:13.979997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746844c4-14b2-4397-af76-5120e8cf1b6f service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:13.982624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d806e-8610-47da-aa93-0330a3f23c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c0f74e2f5794c0189b647e29646eeb2, user_id=8bbc454737084513baa37072cd160e73, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:13.985941 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d806e-8610-47da-aa93-0330a3f23c32 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:13.990794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746844c4-14b2-4397-af76-5120e8cf1b6f service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:13.996125 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddabcf42-5ea6-4697-bb5f-824611737162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:13.997121 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabcf42-5ea6-4697-bb5f-824611737162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:13.997410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabcf42-5ea6-4697-bb5f-824611737162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:13.997864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddabcf42-5ea6-4697-bb5f-824611737162 admin admin] PATH_INFO: `/v3/projects/9b1d323459494d6fa4b206d9aeb403db/users/a2fc16b25b734f11a691a27fd35544e2/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.004088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 816/1607] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:13 2022] PUT /identity/v3/projects/6c0f74e2f5794c0189b647e29646eeb2/users/8bbc454737084513baa37072cd160e73/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:14.005698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d806e-8610-47da-aa93-0330a3f23c32 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:14.016022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabcf42-5ea6-4697-bb5f-824611737162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1d323459494d6fa4b206d9aeb403db, user_id=a2fc16b25b734f11a691a27fd35544e2, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:14.018658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddabcf42-5ea6-4697-bb5f-824611737162 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:14.036646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 792/1608] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:13 2022] PUT /identity/v3/projects/9b1d323459494d6fa4b206d9aeb403db/users/a2fc16b25b734f11a691a27fd35544e2/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:14.038904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47a50e-87e2-40aa-98e9-95e94803960d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.039232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47a50e-87e2-40aa-98e9-95e94803960d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.039513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47a50e-87e2-40aa-98e9-95e94803960d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.064718 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:14.078613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4f47a50e-87e2-40aa-98e9-95e94803960d None None] MFA Rules not processed for user `177b36dc3e014cd388292a75e5424518`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:14.079822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4f47a50e-87e2-40aa-98e9-95e94803960d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:14.096256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d806e-8610-47da-aa93-0330a3f23c32 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:14.170726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 793/1609] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:14.172119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2f09-6891-4546-af86-11f4e9b3514a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.173084 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2f09-6891-4546-af86-11f4e9b3514a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.173381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2f09-6891-4546-af86-11f4e9b3514a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.178845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19fa717-766f-4c5c-802d-7bb6478ce29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:14.180119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fa717-766f-4c5c-802d-7bb6478ce29c admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.183334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fa717-766f-4c5c-802d-7bb6478ce29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.183334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19fa717-766f-4c5c-802d-7bb6478ce29c admin admin] PATH_INFO: `/v3/projects/a21c7dca31124d0daa7f1e1a4594ef85/users/24229eeb077148a99b387427d490a411/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.201988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fa717-766f-4c5c-802d-7bb6478ce29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a21c7dca31124d0daa7f1e1a4594ef85, user_id=24229eeb077148a99b387427d490a411, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:14.201988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19fa717-766f-4c5c-802d-7bb6478ce29c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:14.213213 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e77a2f09-6891-4546-af86-11f4e9b3514a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:14.217741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e77a2f09-6891-4546-af86-11f4e9b3514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:14.224252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 817/1610] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:14 2022] PUT /identity/v3/projects/a21c7dca31124d0daa7f1e1a4594ef85/users/24229eeb077148a99b387427d490a411/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:14.225994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d5af-cd3e-4a15-b703-460f043076a9 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.226688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d5af-cd3e-4a15-b703-460f043076a9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.229556 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79d5af-cd3e-4a15-b703-460f043076a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.267228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-7d79d5af-cd3e-4a15-b703-460f043076a9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:14.268500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7d79d5af-cd3e-4a15-b703-460f043076a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:14.283980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:14.318759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 794/1611] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:14.320411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12046448-9c00-48c8-86b5-d392be033242 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.320888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12046448-9c00-48c8-86b5-d392be033242 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.321246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12046448-9c00-48c8-86b5-d392be033242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.343030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-12046448-9c00-48c8-86b5-d392be033242 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:14.344309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-12046448-9c00-48c8-86b5-d392be033242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:14.361161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 795/1612] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:14.361360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 818/1613] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:14.361469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22990f-eebb-4035-b49b-2ab071c44190 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.361558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79d5af-cd3e-4a15-b703-460f043076a9 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:14.361636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22990f-eebb-4035-b49b-2ab071c44190 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.361636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22990f-eebb-4035-b49b-2ab071c44190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.367720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79d5af-cd3e-4a15-b703-460f043076a9 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:14.373498 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee9101-f13b-435c-b97e-7ae4bed0b4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:14.374939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee9101-f13b-435c-b97e-7ae4bed0b4ac admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.375369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee9101-f13b-435c-b97e-7ae4bed0b4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.375950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee9101-f13b-435c-b97e-7ae4bed0b4ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.376989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee9101-f13b-435c-b97e-7ae4bed0b4ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:14.379155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee9101-f13b-435c-b97e-7ae4bed0b4ac admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:14.393500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fe22990f-eebb-4035-b49b-2ab071c44190 None None] MFA Rules not processed for user `a2fc16b25b734f11a691a27fd35544e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:14.395677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fe22990f-eebb-4035-b49b-2ab071c44190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:14.418856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 819/1614] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:14 2022] POST /identity/v3/projects => generated 363 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:14.421074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff41a61-e2eb-45b9-a96e-4baf6df90aa6 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.424078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff41a61-e2eb-45b9-a96e-4baf6df90aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.424078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff41a61-e2eb-45b9-a96e-4baf6df90aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.460214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-cff41a61-e2eb-45b9-a96e-4baf6df90aa6 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:14.461812 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-cff41a61-e2eb-45b9-a96e-4baf6df90aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:14.493566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 796/1615] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:14.495060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87485854-18e0-4be2-aa32-f56901bc1fd6 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.495388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87485854-18e0-4be2-aa32-f56901bc1fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.495810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87485854-18e0-4be2-aa32-f56901bc1fd6 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.509791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 797/1616] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jul 5 15:13:14 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:13:14.510731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87485854-18e0-4be2-aa32-f56901bc1fd6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:14.512573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:14.537804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 820/1617] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:14.538449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff41a61-e2eb-45b9-a96e-4baf6df90aa6 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:14.587590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87485854-18e0-4be2-aa32-f56901bc1fd6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:14.593804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4c4a1-be22-415e-b0ee-01ca320c096e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:14.595002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4c4a1-be22-415e-b0ee-01ca320c096e service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.595775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4c4a1-be22-415e-b0ee-01ca320c096e service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.596273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4c4a1-be22-415e-b0ee-01ca320c096e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.608717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4c4a1-be22-415e-b0ee-01ca320c096e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:14.610846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4c4a1-be22-415e-b0ee-01ca320c096e service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:14.612106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff41a61-e2eb-45b9-a96e-4baf6df90aa6 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:14.619282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f478449-f979-4da0-a122-9968b4c4820b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:14.619962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478449-f979-4da0-a122-9968b4c4820b service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.620250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478449-f979-4da0-a122-9968b4c4820b service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.620533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f478449-f979-4da0-a122-9968b4c4820b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.624425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 798/1618] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:14.625593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4c4a1-be22-415e-b0ee-01ca320c096e service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:14.636718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4c4a1-be22-415e-b0ee-01ca320c096e service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:14.641592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2df92d-bd07-454c-8574-d452a16acc17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:14.642628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2df92d-bd07-454c-8574-d452a16acc17 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.643659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2df92d-bd07-454c-8574-d452a16acc17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.644191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2df92d-bd07-454c-8574-d452a16acc17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.649124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f478449-f979-4da0-a122-9968b4c4820b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:14.650597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f478449-f979-4da0-a122-9968b4c4820b service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:14.662342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 821/1619] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:14.663230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f478449-f979-4da0-a122-9968b4c4820b service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:14.667794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2df92d-bd07-454c-8574-d452a16acc17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:14.670271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2df92d-bd07-454c-8574-d452a16acc17 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:14.674178 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f478449-f979-4da0-a122-9968b4c4820b service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:14.679962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b277b-1f27-4caf-989d-5ff48cf7846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:14.681488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b277b-1f27-4caf-989d-5ff48cf7846b admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.681973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b277b-1f27-4caf-989d-5ff48cf7846b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.682441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b277b-1f27-4caf-989d-5ff48cf7846b admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.683361 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b277b-1f27-4caf-989d-5ff48cf7846b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:14.685655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b277b-1f27-4caf-989d-5ff48cf7846b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:14.697859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 799/1620] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:14.698907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2df92d-bd07-454c-8574-d452a16acc17 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:14.718283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 822/1621] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:14 2022] POST /identity/v3/users => generated 288 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:14.718283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa72704-1cad-4362-b693-34234f956ef7 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.718283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa72704-1cad-4362-b693-34234f956ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.718283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa72704-1cad-4362-b693-34234f956ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.743562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-7aa72704-1cad-4362-b693-34234f956ef7 None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:14.744799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7aa72704-1cad-4362-b693-34234f956ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:14.794026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2df92d-bd07-454c-8574-d452a16acc17 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:14.799114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579482e6-d7fe-43a6-86d6-7653dca613f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:14.799722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579482e6-d7fe-43a6-86d6-7653dca613f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.800178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579482e6-d7fe-43a6-86d6-7653dca613f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.800567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579482e6-d7fe-43a6-86d6-7653dca613f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:14.816356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:14.901534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579482e6-d7fe-43a6-86d6-7653dca613f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:14.905519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 823/1622] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:14.907161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579482e6-d7fe-43a6-86d6-7653dca613f5 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:14.908467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa72704-1cad-4362-b693-34234f956ef7 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:14.936315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa72704-1cad-4362-b693-34234f956ef7 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:14.937668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 800/1623] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:14.938560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579482e6-d7fe-43a6-86d6-7653dca613f5 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:14.958916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffbcf6-e6db-43c4-8ca5-97f3673ab6a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:14.975010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffbcf6-e6db-43c4-8ca5-97f3673ab6a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:14.975624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffbcf6-e6db-43c4-8ca5-97f3673ab6a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:14.975924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ffbcf6-e6db-43c4-8ca5-97f3673ab6a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:15.013699 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579482e6-d7fe-43a6-86d6-7653dca613f5 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:15.021322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc56199-cdb5-4e1c-9899-89dd5d98e42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:15.022967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc56199-cdb5-4e1c-9899-89dd5d98e42b admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:15.023377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc56199-cdb5-4e1c-9899-89dd5d98e42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:15.023741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc56199-cdb5-4e1c-9899-89dd5d98e42b admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:15.024653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc56199-cdb5-4e1c-9899-89dd5d98e42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:15.026774 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc56199-cdb5-4e1c-9899-89dd5d98e42b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:15.037290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffbcf6-e6db-43c4-8ca5-97f3673ab6a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:15.044493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ffbcf6-e6db-43c4-8ca5-97f3673ab6a5 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:15.047388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 801/1624] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:15.048671 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc56199-cdb5-4e1c-9899-89dd5d98e42b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:15.061460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc56199-cdb5-4e1c-9899-89dd5d98e42b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:15.068128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ada151-d64f-4a97-abd2-aa061a36b906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:15.068810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ada151-d64f-4a97-abd2-aa061a36b906 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:15.069478 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ada151-d64f-4a97-abd2-aa061a36b906 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:15.069776 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ada151-d64f-4a97-abd2-aa061a36b906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:15.079835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 824/1625] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:15.081803 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffbcf6-e6db-43c4-8ca5-97f3673ab6a5 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:15.084360 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ada151-d64f-4a97-abd2-aa061a36b906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:15.087209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ada151-d64f-4a97-abd2-aa061a36b906 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:15.107283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ffbcf6-e6db-43c4-8ca5-97f3673ab6a5 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:15.119090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04c74e-b516-424d-b691-e280b1dc373f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:15.119991 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04c74e-b516-424d-b691-e280b1dc373f admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:15.120277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04c74e-b516-424d-b691-e280b1dc373f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:15.120589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04c74e-b516-424d-b691-e280b1dc373f admin admin] PATH_INFO: `/v3/projects/97923387193048a98b412e6c08866ef5/users/eb61e48984a446498193dcb5e74991fb/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:15.129815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 802/1626] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:15.161496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04c74e-b516-424d-b691-e280b1dc373f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97923387193048a98b412e6c08866ef5, user_id=eb61e48984a446498193dcb5e74991fb, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:15.168152 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04c74e-b516-424d-b691-e280b1dc373f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:15.204558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 825/1627] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:15 2022] PUT /identity/v3/projects/97923387193048a98b412e6c08866ef5/users/eb61e48984a446498193dcb5e74991fb/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:15.209104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ada151-d64f-4a97-abd2-aa061a36b906 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:15.224907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ada151-d64f-4a97-abd2-aa061a36b906 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:15.377481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450213b-cc3f-43c3-b05d-d8b1be07c3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:15.378282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450213b-cc3f-43c3-b05d-d8b1be07c3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:15.378598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450213b-cc3f-43c3-b05d-d8b1be07c3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:15.378960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450213b-cc3f-43c3-b05d-d8b1be07c3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:15.383381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450213b-cc3f-43c3-b05d-d8b1be07c3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:15.386660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450213b-cc3f-43c3-b05d-d8b1be07c3f1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:15.415614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04c74e-b516-424d-b691-e280b1dc373f admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:15.416493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 803/1628] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:15.430042 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04c74e-b516-424d-b691-e280b1dc373f admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:15.441890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578d679-79ee-4e59-8d20-edf773dab229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:15.452035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578d679-79ee-4e59-8d20-edf773dab229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:15.452481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578d679-79ee-4e59-8d20-edf773dab229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:15.452998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578d679-79ee-4e59-8d20-edf773dab229 admin admin] PATH_INFO: `/v3/projects/97923387193048a98b412e6c08866ef5/users/eb61e48984a446498193dcb5e74991fb/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:15.463971 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578d679-79ee-4e59-8d20-edf773dab229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97923387193048a98b412e6c08866ef5, user_id=eb61e48984a446498193dcb5e74991fb, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:15.469963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578d679-79ee-4e59-8d20-edf773dab229 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:15.502051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 826/1629] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:15 2022] PUT /identity/v3/projects/97923387193048a98b412e6c08866ef5/users/eb61e48984a446498193dcb5e74991fb/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:15.503485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626a439-0f5c-4229-a46f-75850314a76c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:15.503958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626a439-0f5c-4229-a46f-75850314a76c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:15.504394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626a439-0f5c-4229-a46f-75850314a76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:15.618287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8626a439-0f5c-4229-a46f-75850314a76c None None] MFA Rules not processed for user `eb61e48984a446498193dcb5e74991fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:15.618287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8626a439-0f5c-4229-a46f-75850314a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:15.853440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 804/1630] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:15.966021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578d679-79ee-4e59-8d20-edf773dab229 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:15.992837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:16.099671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578d679-79ee-4e59-8d20-edf773dab229 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:16.105023 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4236a1a-fd0b-4d2c-a5ff-c2beed7524fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:16.105532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236a1a-fd0b-4d2c-a5ff-c2beed7524fa service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:16.105949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236a1a-fd0b-4d2c-a5ff-c2beed7524fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:16.106337 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4236a1a-fd0b-4d2c-a5ff-c2beed7524fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:16.120823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4236a1a-fd0b-4d2c-a5ff-c2beed7524fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:16.122618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4236a1a-fd0b-4d2c-a5ff-c2beed7524fa service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:16.148248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 827/1631] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:16.399810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626a439-0f5c-4229-a46f-75850314a76c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:16.418336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626a439-0f5c-4229-a46f-75850314a76c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:16.433758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1530ac9-ce71-4338-a000-50b18a662e1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:16.434449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1530ac9-ce71-4338-a000-50b18a662e1b service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:16.434769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1530ac9-ce71-4338-a000-50b18a662e1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:16.435073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1530ac9-ce71-4338-a000-50b18a662e1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:16.443383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:16.582023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1530ac9-ce71-4338-a000-50b18a662e1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:16.586785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1530ac9-ce71-4338-a000-50b18a662e1b service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:16.616528 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 805/1632] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:16.681921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5ee52-838c-4c80-8dcd-218a4e517928 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:16.682210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5ee52-838c-4c80-8dcd-218a4e517928 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:16.682496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c5ee52-838c-4c80-8dcd-218a4e517928 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:16.685903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 828/1633] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jul 5 15:13:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:13:16.704823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110120c-92c0-4f45-8a62-23bda865dce8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:16.705245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110120c-92c0-4f45-8a62-23bda865dce8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:16.705370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110120c-92c0-4f45-8a62-23bda865dce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:16.742622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d110120c-92c0-4f45-8a62-23bda865dce8 None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:16.746421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d110120c-92c0-4f45-8a62-23bda865dce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:16.858816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 806/1634] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jul 5 15:13:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:16.871220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8cb85-a388-4aaf-917b-55d60f86398f None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:16.871807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8cb85-a388-4aaf-917b-55d60f86398f None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:16.872231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8cb85-a388-4aaf-917b-55d60f86398f None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:16.874369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 829/1635] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jul 5 15:13:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:13:16.880294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79213302-d07a-4311-ba68-e3e7fe795ea7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:16.880742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79213302-d07a-4311-ba68-e3e7fe795ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:16.880995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79213302-d07a-4311-ba68-e3e7fe795ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:16.922401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-79213302-d07a-4311-ba68-e3e7fe795ea7 None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:16.923886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-79213302-d07a-4311-ba68-e3e7fe795ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:16.958504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 807/1636] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:13:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:17.003092 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8cb85-a388-4aaf-917b-55d60f86398f None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:17.027793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8cb85-a388-4aaf-917b-55d60f86398f None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:17.034389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34d6d-ac90-409f-b076-4d0fcf7d82c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:17.035626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34d6d-ac90-409f-b076-4d0fcf7d82c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:17.036727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34d6d-ac90-409f-b076-4d0fcf7d82c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:17.037236 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f34d6d-ac90-409f-b076-4d0fcf7d82c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:17.055898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34d6d-ac90-409f-b076-4d0fcf7d82c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:17.058212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f34d6d-ac90-409f-b076-4d0fcf7d82c2 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:17.077080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 830/1637] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:18.934596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5248524-4b94-410b-a3a3-20bc797bc3dd None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:18.935146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5248524-4b94-410b-a3a3-20bc797bc3dd None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:18.935284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5248524-4b94-410b-a3a3-20bc797bc3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:18.995992 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d5248524-4b94-410b-a3a3-20bc797bc3dd None None] MFA Rules not processed for user `24229eeb077148a99b387427d490a411`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:19.002334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d5248524-4b94-410b-a3a3-20bc797bc3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:19.134493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 808/1638] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:19.138936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34d6d-ac90-409f-b076-4d0fcf7d82c2 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:19.337469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f34d6d-ac90-409f-b076-4d0fcf7d82c2 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:19.357341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbc85c-c3f0-4602-83d5-c5aee077c77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:19.358368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbc85c-c3f0-4602-83d5-c5aee077c77a admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:19.358814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbc85c-c3f0-4602-83d5-c5aee077c77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:19.359296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bbc85c-c3f0-4602-83d5-c5aee077c77a admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:19.360182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbc85c-c3f0-4602-83d5-c5aee077c77a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:19.366059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bbc85c-c3f0-4602-83d5-c5aee077c77a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:19.430207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 831/1639] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:19 2022] POST /identity/v3/projects => generated 403 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:13:19.434020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5248524-4b94-410b-a3a3-20bc797bc3dd None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:19.492797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5248524-4b94-410b-a3a3-20bc797bc3dd None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:19.540330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:19.658414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c8368-c0f2-46cc-b22c-6167d5f4d3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:19.660510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c8368-c0f2-46cc-b22c-6167d5f4d3bf admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:19.660915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c8368-c0f2-46cc-b22c-6167d5f4d3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:19.661204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c8368-c0f2-46cc-b22c-6167d5f4d3bf admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:19.661950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c8368-c0f2-46cc-b22c-6167d5f4d3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:19.663980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c8368-c0f2-46cc-b22c-6167d5f4d3bf admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:19.724327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 809/1640] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:19 2022] POST /identity/v3/users => generated 308 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:19.728796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbc85c-c3f0-4602-83d5-c5aee077c77a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:19.749582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bbc85c-c3f0-4602-83d5-c5aee077c77a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:19.754914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bac111-8d55-4f2f-8e85-16de4bc8e9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:19.755889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bac111-8d55-4f2f-8e85-16de4bc8e9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:19.756368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bac111-8d55-4f2f-8e85-16de4bc8e9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:19.756826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bac111-8d55-4f2f-8e85-16de4bc8e9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:19.757752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bac111-8d55-4f2f-8e85-16de4bc8e9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:19.761752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bac111-8d55-4f2f-8e85-16de4bc8e9a3 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:19.789205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c8368-c0f2-46cc-b22c-6167d5f4d3bf admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:19.798704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 832/1641] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:19.803212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c8368-c0f2-46cc-b22c-6167d5f4d3bf admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:19.813175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48f141-b343-43e2-b218-c34cc69badd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:19.815635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48f141-b343-43e2-b218-c34cc69badd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:19.816294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48f141-b343-43e2-b218-c34cc69badd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:19.817917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48f141-b343-43e2-b218-c34cc69badd1 admin admin] PATH_INFO: `/v3/projects/a39f022e0b2e42c5885686ad1e6da8ef/users/bcb7660ad5f24fb8a23984c59d14a6f8/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:19.862445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48f141-b343-43e2-b218-c34cc69badd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39f022e0b2e42c5885686ad1e6da8ef, user_id=bcb7660ad5f24fb8a23984c59d14a6f8, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:19.866876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b48f141-b343-43e2-b218-c34cc69badd1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:19.893685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 810/1642] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:19 2022] PUT /identity/v3/projects/a39f022e0b2e42c5885686ad1e6da8ef/users/bcb7660ad5f24fb8a23984c59d14a6f8/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:13:19.900865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bac111-8d55-4f2f-8e85-16de4bc8e9a3 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:19.920491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bac111-8d55-4f2f-8e85-16de4bc8e9a3 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:20.042378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42342ea7-1b02-43a3-8613-3f0c3e054549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:20.043181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42342ea7-1b02-43a3-8613-3f0c3e054549 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:20.043851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42342ea7-1b02-43a3-8613-3f0c3e054549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:20.043851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42342ea7-1b02-43a3-8613-3f0c3e054549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:20.044779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42342ea7-1b02-43a3-8613-3f0c3e054549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:20.048115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42342ea7-1b02-43a3-8613-3f0c3e054549 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:20.072291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 833/1643] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:20.074311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48f141-b343-43e2-b218-c34cc69badd1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:20.093769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48f141-b343-43e2-b218-c34cc69badd1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:20.113362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab193e-30b2-4c67-a072-2d04a7f049a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:20.116064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab193e-30b2-4c67-a072-2d04a7f049a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:20.117789 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab193e-30b2-4c67-a072-2d04a7f049a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:20.118263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ab193e-30b2-4c67-a072-2d04a7f049a6 admin admin] PATH_INFO: `/v3/projects/a39f022e0b2e42c5885686ad1e6da8ef/users/bcb7660ad5f24fb8a23984c59d14a6f8/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:20.124466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab193e-30b2-4c67-a072-2d04a7f049a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a39f022e0b2e42c5885686ad1e6da8ef, user_id=bcb7660ad5f24fb8a23984c59d14a6f8, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:20.127815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ab193e-30b2-4c67-a072-2d04a7f049a6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:20.147510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 811/1644] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:20 2022] PUT /identity/v3/projects/a39f022e0b2e42c5885686ad1e6da8ef/users/bcb7660ad5f24fb8a23984c59d14a6f8/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:23.070092 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181712-63bb-4e95-8b10-7b86e2ee7f5c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:23.070591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181712-63bb-4e95-8b10-7b86e2ee7f5c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:23.070670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12181712-63bb-4e95-8b10-7b86e2ee7f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:23.131128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:23.167435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-12181712-63bb-4e95-8b10-7b86e2ee7f5c None None] MFA Rules not processed for user `bcb7660ad5f24fb8a23984c59d14a6f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:23.168702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-12181712-63bb-4e95-8b10-7b86e2ee7f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:23.341618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 834/1645] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:23 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:23.401324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab193e-30b2-4c67-a072-2d04a7f049a6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:23.564693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ab193e-30b2-4c67-a072-2d04a7f049a6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:23.576611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:23.577869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:23.578277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:23.578655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] PATH_INFO: `/v3/users/177b36dc3e014cd388292a75e5424518` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:23.581833 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=177b36dc3e014cd388292a75e5424518)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:23.583590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: WARNING py.warnings [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:13:23.583590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: warnings.warn(msg) Jul 05 15:13:23.583590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: Jul 05 15:13:23.589320 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:23.745509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '177b36dc3e014cd388292a75e5424518'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:23.746500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '177b36dc3e014cd388292a75e5424518'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:23.749466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '177b36dc3e014cd388292a75e5424518'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:23.776708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '177b36dc3e014cd388292a75e5424518'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:23.812340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '177b36dc3e014cd388292a75e5424518'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:23.833977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:23.872115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 812/1646] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:13:23 2022] DELETE /identity/v3/users/177b36dc3e014cd388292a75e5424518 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:23.877796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21def7-e026-45b2-bb61-3f26f98dbd28 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:23.878095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21def7-e026-45b2-bb61-3f26f98dbd28 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:23.878383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21def7-e026-45b2-bb61-3f26f98dbd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:23.909342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ff21def7-e026-45b2-bb61-3f26f98dbd28 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:23.913479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ff21def7-e026-45b2-bb61-3f26f98dbd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:24.012713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 835/1647] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:24.035955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:24.038786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:24.184078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673d3db-51a3-4d03-b7a9-e507a6a4aa11 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:24.194412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855185b2-7ac4-4a12-99f6-7e6c8ba32fe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:24.195428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855185b2-7ac4-4a12-99f6-7e6c8ba32fe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:24.195824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855185b2-7ac4-4a12-99f6-7e6c8ba32fe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:24.196203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855185b2-7ac4-4a12-99f6-7e6c8ba32fe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:24.219931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855185b2-7ac4-4a12-99f6-7e6c8ba32fe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:24.225128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855185b2-7ac4-4a12-99f6-7e6c8ba32fe2 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:24.249501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 813/1648] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:25.464025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87518369-e10d-4662-bab8-8186677dab2a None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:25.464519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87518369-e10d-4662-bab8-8186677dab2a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:25.464597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87518369-e10d-4662-bab8-8186677dab2a None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:25.465994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 836/1649] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:13:25.479005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d444f75-7813-4de6-8eb1-17304edc1737 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:25.479517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d444f75-7813-4de6-8eb1-17304edc1737 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:25.480157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d444f75-7813-4de6-8eb1-17304edc1737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:25.522036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0d444f75-7813-4de6-8eb1-17304edc1737 None None] MFA Rules not processed for user `c68f1297771e4c03a897dd1f0ac21cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:25.524157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0d444f75-7813-4de6-8eb1-17304edc1737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:25.699001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 814/1650] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:13:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:25.852764 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb085-ed21-4fd6-90a6-9735e6c8891a None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:25.852892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb085-ed21-4fd6-90a6-9735e6c8891a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:25.853386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01eb085-ed21-4fd6-90a6-9735e6c8891a None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:25.854984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 837/1651] 10.222.0.23 () {54 vars in 960 bytes} [Tue Jul 5 15:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 05 15:13:25.868375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe3829-c2b8-43b4-b4f0-a9c1b6ae89c7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:25.870547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe3829-c2b8-43b4-b4f0-a9c1b6ae89c7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:25.870874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe3829-c2b8-43b4-b4f0-a9c1b6ae89c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:25.917551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5efe3829-c2b8-43b4-b4f0-a9c1b6ae89c7 None None] MFA Rules not processed for user `c68f1297771e4c03a897dd1f0ac21cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:25.920321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5efe3829-c2b8-43b4-b4f0-a9c1b6ae89c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:25.934703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 815/1652] 10.222.0.23 () {58 vars in 1057 bytes} [Tue Jul 5 15:13:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:25.945099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eb085-ed21-4fd6-90a6-9735e6c8891a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:25.979143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01eb085-ed21-4fd6-90a6-9735e6c8891a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:25.991851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acb2f1-1c1d-48e2-aef9-85339ffc59fb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:25.992863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acb2f1-1c1d-48e2-aef9-85339ffc59fb service placement] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:25.993395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acb2f1-1c1d-48e2-aef9-85339ffc59fb service placement] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:25.993869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acb2f1-1c1d-48e2-aef9-85339ffc59fb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:26.012323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acb2f1-1c1d-48e2-aef9-85339ffc59fb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:26.054079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acb2f1-1c1d-48e2-aef9-85339ffc59fb service placement] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:26.117829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 838/1653] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:26.563113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe3829-c2b8-43b4-b4f0-a9c1b6ae89c7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:26.569139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5efe3829-c2b8-43b4-b4f0-a9c1b6ae89c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:26.606643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe3829-c2b8-43b4-b4f0-a9c1b6ae89c7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:26.630584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:26.631917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:26.632525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:26.633009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] PATH_INFO: `/v3/projects/7ccd2398b3334bac9f0b3e188cb85324` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:26.634649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ccd2398b3334bac9f0b3e188cb85324)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:26.638054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:26.720305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ccd2398b3334bac9f0b3e188cb85324', 'request_id': 'req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:26.730244 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5678-9748-4eed-bc9a-f526d582f355 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:26.734550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5678-9748-4eed-bc9a-f526d582f355 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:26.735391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e5678-9748-4eed-bc9a-f526d582f355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:26.739137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ccd2398b3334bac9f0b3e188cb85324', 'request_id': 'req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:26.759247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ccd2398b3334bac9f0b3e188cb85324', 'request_id': 'req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:26.799124 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] The token cache is being invalidate because project 7ccd2398b3334bac9f0b3e188cb85324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:13:26.806362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:26.813965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:26.823032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 816/1654] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:13:26 2022] DELETE /identity/v3/projects/7ccd2398b3334bac9f0b3e188cb85324 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:26.826643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:26.830725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:26.831429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-728e5678-9748-4eed-bc9a-f526d582f355 None None] MFA Rules not processed for user `0187789947064682a8d9e685c2aadb9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:26.834450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-728e5678-9748-4eed-bc9a-f526d582f355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:26.854682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:26.943904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a50217f-0a0a-4f72-a9da-0afbed05fa1e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:26.950572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:26.952809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:26.953298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:26.954641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] PATH_INFO: `/v3/users/eb61e48984a446498193dcb5e74991fb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:26.956558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb61e48984a446498193dcb5e74991fb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:26.959597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:26.990812 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 839/1655] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:27.007563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e5678-9748-4eed-bc9a-f526d582f355 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:27.009376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-728e5678-9748-4eed-bc9a-f526d582f355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:27.048417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:27.051729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb61e48984a446498193dcb5e74991fb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:27.054221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb61e48984a446498193dcb5e74991fb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:27.055823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb61e48984a446498193dcb5e74991fb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:27.083551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb61e48984a446498193dcb5e74991fb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:27.102401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dad10afc-ea56-46ba-93c5-356cc0791c23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb61e48984a446498193dcb5e74991fb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:27.191263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 817/1656] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:13:26 2022] DELETE /identity/v3/users/eb61e48984a446498193dcb5e74991fb => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:27.195005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf429d0-2fa4-49ba-841b-e954a95f2773 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:27.195781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf429d0-2fa4-49ba-841b-e954a95f2773 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:27.197242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf429d0-2fa4-49ba-841b-e954a95f2773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:27.256397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e5678-9748-4eed-bc9a-f526d582f355 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:27.297380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-edf429d0-2fa4-49ba-841b-e954a95f2773 None None] MFA Rules not processed for user `ef098bbb9ee94bf4aadb4b59516890fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:27.298902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-edf429d0-2fa4-49ba-841b-e954a95f2773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:27.433169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 818/1657] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:27.458222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1527f-31b1-49af-a2e8-dabc6e4b84bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:27.459315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1527f-31b1-49af-a2e8-dabc6e4b84bf service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:27.459800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1527f-31b1-49af-a2e8-dabc6e4b84bf service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:27.460277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1527f-31b1-49af-a2e8-dabc6e4b84bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:27.478963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-81a1527f-31b1-49af-a2e8-dabc6e4b84bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:27.624795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf429d0-2fa4-49ba-841b-e954a95f2773 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:27.627548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-edf429d0-2fa4-49ba-841b-e954a95f2773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:27.684995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf429d0-2fa4-49ba-841b-e954a95f2773 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:27.712183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24419c2d-9ac3-4e27-a83d-e863c09d506f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:27.713162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24419c2d-9ac3-4e27-a83d-e863c09d506f service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:27.713451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24419c2d-9ac3-4e27-a83d-e863c09d506f service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:27.713731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24419c2d-9ac3-4e27-a83d-e863c09d506f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:27.727243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1527f-31b1-49af-a2e8-dabc6e4b84bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:27.728959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1527f-31b1-49af-a2e8-dabc6e4b84bf service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:27.739888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24419c2d-9ac3-4e27-a83d-e863c09d506f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:27.749465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 840/1658] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 742 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:13:27.749996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24419c2d-9ac3-4e27-a83d-e863c09d506f service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:27.798448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 819/1659] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:13:28.555533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1527f-31b1-49af-a2e8-dabc6e4b84bf service nova] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:28.557019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-81a1527f-31b1-49af-a2e8-dabc6e4b84bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:28.665208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:28.704527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1527f-31b1-49af-a2e8-dabc6e4b84bf service nova] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:28.710054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:28.711733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:28.711733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:28.712216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] PATH_INFO: `/v3/users/24229eeb077148a99b387427d490a411` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:28.713372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] RBAC: Authorizing `identity:delete_user(user_id=24229eeb077148a99b387427d490a411)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:28.715563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:28.731776 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24419c2d-9ac3-4e27-a83d-e863c09d506f service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:28.733838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-24419c2d-9ac3-4e27-a83d-e863c09d506f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:28.758233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:28.829551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24419c2d-9ac3-4e27-a83d-e863c09d506f service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:28.857684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:28.859152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:28.860530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24229eeb077148a99b387427d490a411'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:28.861771 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24229eeb077148a99b387427d490a411'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:28.862617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24229eeb077148a99b387427d490a411'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:28.872953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:28.873250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] PATH_INFO: `/v3/projects/97923387193048a98b412e6c08866ef5` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:28.874693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] RBAC: Authorizing `identity:delete_project(project_id=97923387193048a98b412e6c08866ef5)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:28.876700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:28.881228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24229eeb077148a99b387427d490a411'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:28.903430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9a2fa790-64f6-4bbf-bacb-7a0b2328431f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24229eeb077148a99b387427d490a411'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:28.937581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 841/1660] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:13:28 2022] DELETE /identity/v3/users/24229eeb077148a99b387427d490a411 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:29.037312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97923387193048a98b412e6c08866ef5', 'request_id': 'req-0745d6f0-9913-4729-a456-036391e8e52e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:29.062846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97923387193048a98b412e6c08866ef5', 'request_id': 'req-0745d6f0-9913-4729-a456-036391e8e52e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:29.070677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97923387193048a98b412e6c08866ef5', 'request_id': 'req-0745d6f0-9913-4729-a456-036391e8e52e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:29.101928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] The token cache is being invalidate because project 97923387193048a98b412e6c08866ef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:13:29.102238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:29.105942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0745d6f0-9913-4729-a456-036391e8e52e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:29.112572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 820/1661] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:13:28 2022] DELETE /identity/v3/projects/97923387193048a98b412e6c08866ef5 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:29.114661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70a3de-8da7-4de6-a9b1-47a45c9f26ef None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:29.114763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70a3de-8da7-4de6-a9b1-47a45c9f26ef None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:29.115028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d70a3de-8da7-4de6-a9b1-47a45c9f26ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:29.141667 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-6d70a3de-8da7-4de6-a9b1-47a45c9f26ef None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:29.143351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6d70a3de-8da7-4de6-a9b1-47a45c9f26ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:29.374037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 842/1662] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:29.382479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755e7a4-d273-4ea6-b2b5-c0735f905b1a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:29.383186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755e7a4-d273-4ea6-b2b5-c0735f905b1a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:29.383575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755e7a4-d273-4ea6-b2b5-c0735f905b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:29.444278 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0755e7a4-d273-4ea6-b2b5-c0735f905b1a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:29.446514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0755e7a4-d273-4ea6-b2b5-c0735f905b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:29.500355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 821/1663] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:29.506712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70a3de-8da7-4de6-a9b1-47a45c9f26ef None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:29.549254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d70a3de-8da7-4de6-a9b1-47a45c9f26ef None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:29.571445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f416b-4937-4b21-b422-3aae79487c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:29.572401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f416b-4937-4b21-b422-3aae79487c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:29.573869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f416b-4937-4b21-b422-3aae79487c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:29.574641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f416b-4937-4b21-b422-3aae79487c3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:29.577596 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f416b-4937-4b21-b422-3aae79487c3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:29.578076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f416b-4937-4b21-b422-3aae79487c3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:29.593948 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f416b-4937-4b21-b422-3aae79487c3f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:29.649468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 843/1664] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:29.654051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755e7a4-d273-4ea6-b2b5-c0735f905b1a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:29.669660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755e7a4-d273-4ea6-b2b5-c0735f905b1a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:29.693058 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc430b3-94bf-4ef9-b181-fbe8511c0c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:29.702270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc430b3-94bf-4ef9-b181-fbe8511c0c5a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:29.702635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc430b3-94bf-4ef9-b181-fbe8511c0c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:29.702981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc430b3-94bf-4ef9-b181-fbe8511c0c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:29.703764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc430b3-94bf-4ef9-b181-fbe8511c0c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:29.710477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc430b3-94bf-4ef9-b181-fbe8511c0c5a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:29.744679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f416b-4937-4b21-b422-3aae79487c3f admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:29.745421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-893f416b-4937-4b21-b422-3aae79487c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:29.763410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f416b-4937-4b21-b422-3aae79487c3f admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:29.791782 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:29.798412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:29.798737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:29.799030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] PATH_INFO: `/v3/users/0036ce88dc5247bc83f35de8bf8e68b1` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:29.808594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0036ce88dc5247bc83f35de8bf8e68b1)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:29.811814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 822/1665] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:29 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:29.812784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc430b3-94bf-4ef9-b181-fbe8511c0c5a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:29.816022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:29.974983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0036ce88dc5247bc83f35de8bf8e68b1'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:29.976973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0036ce88dc5247bc83f35de8bf8e68b1'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:29.978203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0036ce88dc5247bc83f35de8bf8e68b1'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:29.996839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:30.006992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0036ce88dc5247bc83f35de8bf8e68b1'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:30.029550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e6152e81-3c32-40b8-8065-72123da40153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0036ce88dc5247bc83f35de8bf8e68b1'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:30.058609 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc430b3-94bf-4ef9-b181-fbe8511c0c5a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:30.089243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069e4d6-e811-4964-ad99-cd789a2e7923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:30.094139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069e4d6-e811-4964-ad99-cd789a2e7923 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:30.094439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069e4d6-e811-4964-ad99-cd789a2e7923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:30.094726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069e4d6-e811-4964-ad99-cd789a2e7923 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:30.095499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069e4d6-e811-4964-ad99-cd789a2e7923 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:30.097565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069e4d6-e811-4964-ad99-cd789a2e7923 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:30.117091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 844/1666] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:13:29 2022] DELETE /identity/v3/users/0036ce88dc5247bc83f35de8bf8e68b1 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:30.118655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85589ca2-54b3-4437-8e8e-27521f65437e None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:30.119714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85589ca2-54b3-4437-8e8e-27521f65437e None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:30.120796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85589ca2-54b3-4437-8e8e-27521f65437e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:30.171710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 823/1667] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:29 2022] POST /identity/v3/users => generated 284 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:30.178092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069e4d6-e811-4964-ad99-cd789a2e7923 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:30.188993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2069e4d6-e811-4964-ad99-cd789a2e7923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:30.221968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-85589ca2-54b3-4437-8e8e-27521f65437e None None] MFA Rules not processed for user `8bbc454737084513baa37072cd160e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:30.223797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-85589ca2-54b3-4437-8e8e-27521f65437e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:30.227823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069e4d6-e811-4964-ad99-cd789a2e7923 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:30.367015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:30.403470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:30.406131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:30.407643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:30.408700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] PATH_INFO: `/v3/projects/a21c7dca31124d0daa7f1e1a4594ef85` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:30.411666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a21c7dca31124d0daa7f1e1a4594ef85)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:30.415611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:30.434180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 845/1668] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:30.434180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3fe02-f506-4495-b8b5-4a63ffd17d24 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:30.434180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3fe02-f506-4495-b8b5-4a63ffd17d24 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:30.434180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3fe02-f506-4495-b8b5-4a63ffd17d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:30.487969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-47f3fe02-f506-4495-b8b5-4a63ffd17d24 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:30.489821 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-47f3fe02-f506-4495-b8b5-4a63ffd17d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:30.509391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 846/1669] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:30.509391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3fe02-f506-4495-b8b5-4a63ffd17d24 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:30.511228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-47f3fe02-f506-4495-b8b5-4a63ffd17d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:30.539754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3fe02-f506-4495-b8b5-4a63ffd17d24 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:30.554468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea08ff1-b5eb-43a9-8045-eaacb2da624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:30.555308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea08ff1-b5eb-43a9-8045-eaacb2da624c admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:30.555599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea08ff1-b5eb-43a9-8045-eaacb2da624c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:30.555879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea08ff1-b5eb-43a9-8045-eaacb2da624c admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:30.556623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea08ff1-b5eb-43a9-8045-eaacb2da624c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:30.558509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea08ff1-b5eb-43a9-8045-eaacb2da624c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:30.585185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 847/1670] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:30.587205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a21c7dca31124d0daa7f1e1a4594ef85', 'request_id': 'req-4a5aff46-5f27-4af0-8b46-be36dc927109'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:30.591754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea08ff1-b5eb-43a9-8045-eaacb2da624c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:30.596309 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-cea08ff1-b5eb-43a9-8045-eaacb2da624c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:30.603517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a21c7dca31124d0daa7f1e1a4594ef85', 'request_id': 'req-4a5aff46-5f27-4af0-8b46-be36dc927109'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:30.618056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a21c7dca31124d0daa7f1e1a4594ef85', 'request_id': 'req-4a5aff46-5f27-4af0-8b46-be36dc927109'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:30.640833 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] The token cache is being invalidate because project a21c7dca31124d0daa7f1e1a4594ef85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:13:30.642450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:30.643370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:30.648352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 824/1671] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:13:30 2022] DELETE /identity/v3/projects/a21c7dca31124d0daa7f1e1a4594ef85 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:30.649507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:30.652327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:30.864982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5aff46-5f27-4af0-8b46-be36dc927109 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:30.877494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981327c5-6662-4455-957d-44ddc502baf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:30.878353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981327c5-6662-4455-957d-44ddc502baf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:30.878756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981327c5-6662-4455-957d-44ddc502baf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:30.879168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981327c5-6662-4455-957d-44ddc502baf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:30.881188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981327c5-6662-4455-957d-44ddc502baf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:30.883161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981327c5-6662-4455-957d-44ddc502baf3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:30.907734 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 825/1672] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:30.911954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981327c5-6662-4455-957d-44ddc502baf3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:30.919481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-981327c5-6662-4455-957d-44ddc502baf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:30.955057 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea08ff1-b5eb-43a9-8045-eaacb2da624c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:30.959219 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981327c5-6662-4455-957d-44ddc502baf3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:30.978086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:30.979164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:30.979771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:30.979979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] PATH_INFO: `/v3/users/bcb7660ad5f24fb8a23984c59d14a6f8` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:30.982812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb7660ad5f24fb8a23984c59d14a6f8)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:30.985386 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:31.008598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:31.098567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66354411-a62b-4ec5-bb42-4693183930d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:31.099299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66354411-a62b-4ec5-bb42-4693183930d8 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:31.099642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66354411-a62b-4ec5-bb42-4693183930d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:31.099927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66354411-a62b-4ec5-bb42-4693183930d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:31.103778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb7660ad5f24fb8a23984c59d14a6f8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:31.104840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb7660ad5f24fb8a23984c59d14a6f8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:31.105638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb7660ad5f24fb8a23984c59d14a6f8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:31.115756 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-66354411-a62b-4ec5-bb42-4693183930d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:31.117628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb7660ad5f24fb8a23984c59d14a6f8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:31.134673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb7660ad5f24fb8a23984c59d14a6f8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:31.153711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:31.213506 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 826/1673] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:13:30 2022] DELETE /identity/v3/users/bcb7660ad5f24fb8a23984c59d14a6f8 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:31.214718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:31.218937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:31.287503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66354411-a62b-4ec5-bb42-4693183930d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:31.289878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66354411-a62b-4ec5-bb42-4693183930d8 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:31.311469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 848/1674] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:31.312660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66354411-a62b-4ec5-bb42-4693183930d8 service nova] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:31.346674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66354411-a62b-4ec5-bb42-4693183930d8 service nova] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:31.384762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd821710-df2e-4d7c-aa64-671f118f8ca7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:31.395684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1b672-6277-4c53-a504-fa86c049730b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:31.396356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1b672-6277-4c53-a504-fa86c049730b service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:31.396638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1b672-6277-4c53-a504-fa86c049730b service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:31.396913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1b672-6277-4c53-a504-fa86c049730b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:31.402605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-34f1b672-6277-4c53-a504-fa86c049730b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:31.516412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569029a8-f906-4445-8b05-1b2c22b45247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:31.517401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569029a8-f906-4445-8b05-1b2c22b45247 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:31.517605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569029a8-f906-4445-8b05-1b2c22b45247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:31.517883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569029a8-f906-4445-8b05-1b2c22b45247 admin admin] PATH_INFO: `/v3/projects/c08409687cb64fa4971e1bae473c9230/users/eadc03a40546486585def15c075da201/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:31.562015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569029a8-f906-4445-8b05-1b2c22b45247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c08409687cb64fa4971e1bae473c9230, user_id=eadc03a40546486585def15c075da201, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:31.568759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569029a8-f906-4445-8b05-1b2c22b45247 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:31.606410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 849/1675] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:31 2022] PUT /identity/v3/projects/c08409687cb64fa4971e1bae473c9230/users/eadc03a40546486585def15c075da201/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:31.616842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedc50d-9a6a-474f-90f5-81868351c5e3 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:31.617304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedc50d-9a6a-474f-90f5-81868351c5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:31.618039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dedc50d-9a6a-474f-90f5-81868351c5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:31.664349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1b672-6277-4c53-a504-fa86c049730b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:31.671391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f1b672-6277-4c53-a504-fa86c049730b service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:31.724186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 827/1676] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:31.730204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-8dedc50d-9a6a-474f-90f5-81868351c5e3 None None] MFA Rules not processed for user `eadc03a40546486585def15c075da201`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:31.731867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8dedc50d-9a6a-474f-90f5-81868351c5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:31.907799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:31.962620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 850/1677] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:31.963982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1b672-6277-4c53-a504-fa86c049730b service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:31.965536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-34f1b672-6277-4c53-a504-fa86c049730b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:31.984763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1b672-6277-4c53-a504-fa86c049730b service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:32.145745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bddd4e-fff8-4ac4-a326-0f49f1fa8488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:32.146447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bddd4e-fff8-4ac4-a326-0f49f1fa8488 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:32.147212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bddd4e-fff8-4ac4-a326-0f49f1fa8488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:32.147674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bddd4e-fff8-4ac4-a326-0f49f1fa8488 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:32.148532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bddd4e-fff8-4ac4-a326-0f49f1fa8488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:32.151215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bddd4e-fff8-4ac4-a326-0f49f1fa8488 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:32.218575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 828/1678] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:31 2022] POST /identity/v3/projects => generated 369 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:32.222494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedc50d-9a6a-474f-90f5-81868351c5e3 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:32.252399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dedc50d-9a6a-474f-90f5-81868351c5e3 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:32.361086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bddd4e-fff8-4ac4-a326-0f49f1fa8488 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:32.364260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-55bddd4e-fff8-4ac4-a326-0f49f1fa8488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:32.405597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:32.424965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c4faf-4ee3-4ecc-9f31-37e1e6ad42c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:32.425903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c4faf-4ee3-4ecc-9f31-37e1e6ad42c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:32.427358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c4faf-4ee3-4ecc-9f31-37e1e6ad42c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:32.427788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c4faf-4ee3-4ecc-9f31-37e1e6ad42c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:32.429442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c4faf-4ee3-4ecc-9f31-37e1e6ad42c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:32.435785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c4faf-4ee3-4ecc-9f31-37e1e6ad42c0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:32.484596 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 851/1679] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:32 2022] POST /identity/v3/users => generated 291 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:32.488504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f194-fb80-4e5b-97ea-30c914c10065 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:32.490424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f194-fb80-4e5b-97ea-30c914c10065 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:32.490790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01f194-fb80-4e5b-97ea-30c914c10065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:32.541460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-3d01f194-fb80-4e5b-97ea-30c914c10065 None None] MFA Rules not processed for user `ba0f3ef2b024464a9c3e71f5d639ca69`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:32.543614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3d01f194-fb80-4e5b-97ea-30c914c10065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:32.548958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bddd4e-fff8-4ac4-a326-0f49f1fa8488 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:32.563092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:32.564072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:32.564465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:32.564840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] PATH_INFO: `/v3/projects/a39f022e0b2e42c5885686ad1e6da8ef` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:32.566492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a39f022e0b2e42c5885686ad1e6da8ef)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:32.571307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:32.662974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a39f022e0b2e42c5885686ad1e6da8ef', 'request_id': 'req-d6dc1efa-7488-413e-bea8-1e554459005f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:32.678937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a39f022e0b2e42c5885686ad1e6da8ef', 'request_id': 'req-d6dc1efa-7488-413e-bea8-1e554459005f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:32.694834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a39f022e0b2e42c5885686ad1e6da8ef', 'request_id': 'req-d6dc1efa-7488-413e-bea8-1e554459005f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:32.703488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 852/1680] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:32.705043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f194-fb80-4e5b-97ea-30c914c10065 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:32.724276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] The token cache is being invalidate because project a39f022e0b2e42c5885686ad1e6da8ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:13:32.751248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:32.752248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d6dc1efa-7488-413e-bea8-1e554459005f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:32.755383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 829/1681] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:13:32 2022] DELETE /identity/v3/projects/a39f022e0b2e42c5885686ad1e6da8ef => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:32.757504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e62b9-1602-47b8-959b-2b9c145d663a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:32.760560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e62b9-1602-47b8-959b-2b9c145d663a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:32.761711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e62b9-1602-47b8-959b-2b9c145d663a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:32.779081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01f194-fb80-4e5b-97ea-30c914c10065 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:32.817930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-602e62b9-1602-47b8-959b-2b9c145d663a None None] MFA Rules not processed for user `ba0f3ef2b024464a9c3e71f5d639ca69`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:32.820255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-602e62b9-1602-47b8-959b-2b9c145d663a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:33.005956 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cefdc9-0cdb-46cb-aa61-c31b0453f309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:33.011848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cefdc9-0cdb-46cb-aa61-c31b0453f309 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:33.012473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cefdc9-0cdb-46cb-aa61-c31b0453f309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:33.013891 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cefdc9-0cdb-46cb-aa61-c31b0453f309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:33.015561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cefdc9-0cdb-46cb-aa61-c31b0453f309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:33.018222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cefdc9-0cdb-46cb-aa61-c31b0453f309 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:33.051902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 853/1682] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:33.054025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368e799-909c-4fc9-b768-3a047e139b7a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:33.056729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368e799-909c-4fc9-b768-3a047e139b7a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:33.057343 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368e799-909c-4fc9-b768-3a047e139b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:33.064447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 830/1683] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:33.065887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e62b9-1602-47b8-959b-2b9c145d663a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:33.071686 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-602e62b9-1602-47b8-959b-2b9c145d663a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:33.074991 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:33.117202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-a368e799-909c-4fc9-b768-3a047e139b7a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:33.121986 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a368e799-909c-4fc9-b768-3a047e139b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:33.130245 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e62b9-1602-47b8-959b-2b9c145d663a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:33.146237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53915c4-de90-47ee-b596-a0b6f23c209b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:33.149799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53915c4-de90-47ee-b596-a0b6f23c209b admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:33.151078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53915c4-de90-47ee-b596-a0b6f23c209b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:33.151644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 854/1684] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:33.153681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53915c4-de90-47ee-b596-a0b6f23c209b admin admin] PATH_INFO: `/v3/projects/cfdc3334eeb94e6b8c59dbe56d15f0bf/users/cf40b4f00b3d41439c87967451e24661/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:33.155122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368e799-909c-4fc9-b768-3a047e139b7a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:33.160513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a368e799-909c-4fc9-b768-3a047e139b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:33.203404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53915c4-de90-47ee-b596-a0b6f23c209b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfdc3334eeb94e6b8c59dbe56d15f0bf, user_id=cf40b4f00b3d41439c87967451e24661, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:33.212316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53915c4-de90-47ee-b596-a0b6f23c209b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:33.246685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 831/1685] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:33 2022] PUT /identity/v3/projects/cfdc3334eeb94e6b8c59dbe56d15f0bf/users/cf40b4f00b3d41439c87967451e24661/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:33.252011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634dd61-4d54-4538-b404-4803465cff85 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:33.252492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634dd61-4d54-4538-b404-4803465cff85 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:33.254912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634dd61-4d54-4538-b404-4803465cff85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:33.317547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d634dd61-4d54-4538-b404-4803465cff85 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:33.320234 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d634dd61-4d54-4538-b404-4803465cff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:33.401266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:33.416884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368e799-909c-4fc9-b768-3a047e139b7a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:33.510459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 832/1686] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:33.513822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634dd61-4d54-4538-b404-4803465cff85 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:33.541086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634dd61-4d54-4538-b404-4803465cff85 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:33.559893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caec8e8-3945-4da7-9db6-39bb2a2aa5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:33.564133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caec8e8-3945-4da7-9db6-39bb2a2aa5ab admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:33.564133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caec8e8-3945-4da7-9db6-39bb2a2aa5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:33.564133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caec8e8-3945-4da7-9db6-39bb2a2aa5ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:33.564430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caec8e8-3945-4da7-9db6-39bb2a2aa5ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:33.578915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caec8e8-3945-4da7-9db6-39bb2a2aa5ab admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:33.613440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 833/1687] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:33.615272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caec8e8-3945-4da7-9db6-39bb2a2aa5ab admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:33.628894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6caec8e8-3945-4da7-9db6-39bb2a2aa5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:33.647828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26cf0d-c30f-4beb-a754-814c10d2ade1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:33.648799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26cf0d-c30f-4beb-a754-814c10d2ade1 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:33.649711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26cf0d-c30f-4beb-a754-814c10d2ade1 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:33.650101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26cf0d-c30f-4beb-a754-814c10d2ade1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:33.655944 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-df26cf0d-c30f-4beb-a754-814c10d2ade1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:33.675808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caec8e8-3945-4da7-9db6-39bb2a2aa5ab admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:33.686903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:33.689627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:33.690010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:33.690366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] PATH_INFO: `/v3/projects/542b790e5cc5402fb8a1438496a7fc5a` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:33.694405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=542b790e5cc5402fb8a1438496a7fc5a)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:33.698790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:33.822824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '542b790e5cc5402fb8a1438496a7fc5a', 'request_id': 'req-fc86f384-638b-4431-8a09-3f50a76d4673'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:33.842119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '542b790e5cc5402fb8a1438496a7fc5a', 'request_id': 'req-fc86f384-638b-4431-8a09-3f50a76d4673'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:33.853349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26cf0d-c30f-4beb-a754-814c10d2ade1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:33.855619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26cf0d-c30f-4beb-a754-814c10d2ade1 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:33.859422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '542b790e5cc5402fb8a1438496a7fc5a', 'request_id': 'req-fc86f384-638b-4431-8a09-3f50a76d4673'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:33.889090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] The token cache is being invalidate because project 542b790e5cc5402fb8a1438496a7fc5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:13:33.889585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:33.890280 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:33.896930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 834/1688] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:13:33 2022] DELETE /identity/v3/projects/542b790e5cc5402fb8a1438496a7fc5a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:33.896930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:33.900251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:33.914917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 855/1689] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 762 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:33.929896 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26cf0d-c30f-4beb-a754-814c10d2ade1 service nova] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:33.981302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26cf0d-c30f-4beb-a754-814c10d2ade1 service nova] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:34.137272 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:34.246800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86f384-638b-4431-8a09-3f50a76d4673 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:34.269693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1f9e6-f7a5-4f6b-969b-a0d2eaf5b2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:34.271395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1f9e6-f7a5-4f6b-969b-a0d2eaf5b2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:34.276188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1f9e6-f7a5-4f6b-969b-a0d2eaf5b2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:34.276979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1f9e6-f7a5-4f6b-969b-a0d2eaf5b2a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:34.279684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1f9e6-f7a5-4f6b-969b-a0d2eaf5b2a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:34.280755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1f9e6-f7a5-4f6b-969b-a0d2eaf5b2a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:34.282518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1f9e6-f7a5-4f6b-969b-a0d2eaf5b2a2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:34.288404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38be1a-7d18-46a5-8d0c-73bba6613537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:34.297292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38be1a-7d18-46a5-8d0c-73bba6613537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:34.297626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38be1a-7d18-46a5-8d0c-73bba6613537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:34.297914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38be1a-7d18-46a5-8d0c-73bba6613537 admin admin] PATH_INFO: `/v3/projects/cfdc3334eeb94e6b8c59dbe56d15f0bf/users/cf40b4f00b3d41439c87967451e24661/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:34.311010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38be1a-7d18-46a5-8d0c-73bba6613537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfdc3334eeb94e6b8c59dbe56d15f0bf, user_id=cf40b4f00b3d41439c87967451e24661, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:34.325179 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38be1a-7d18-46a5-8d0c-73bba6613537 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:34.333864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 835/1690] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:34.344759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a9df4-b939-498f-974e-78b254c369d5 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:34.345174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a9df4-b939-498f-974e-78b254c369d5 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:34.346507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a9df4-b939-498f-974e-78b254c369d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:34.376875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 856/1691] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:33 2022] PUT /identity/v3/projects/cfdc3334eeb94e6b8c59dbe56d15f0bf/users/cf40b4f00b3d41439c87967451e24661/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:34.380021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38be1a-7d18-46a5-8d0c-73bba6613537 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:34.415673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4a9a9df4-b939-498f-974e-78b254c369d5 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:34.416962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4a9a9df4-b939-498f-974e-78b254c369d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:34.535497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:34.548711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38be1a-7d18-46a5-8d0c-73bba6613537 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:34.562094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a5ba4-467e-4f5d-a670-659fc213cbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:34.562863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a5ba4-467e-4f5d-a670-659fc213cbc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:34.563262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a5ba4-467e-4f5d-a670-659fc213cbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:34.563608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a5ba4-467e-4f5d-a670-659fc213cbc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:34.564386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a5ba4-467e-4f5d-a670-659fc213cbc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:34.566396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a5ba4-467e-4f5d-a670-659fc213cbc7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:34.627756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 836/1692] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:34.631027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f136b8-f38d-4559-9eac-6d6870dbc5d4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:34.631377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f136b8-f38d-4559-9eac-6d6870dbc5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:34.632882 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f136b8-f38d-4559-9eac-6d6870dbc5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:34.663467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 857/1693] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:34 2022] POST /identity/v3/projects => generated 373 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:34.666705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3cd1-23fd-4052-b0c8-a8f888376737 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:34.667221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3cd1-23fd-4052-b0c8-a8f888376737 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:34.667608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe3cd1-23fd-4052-b0c8-a8f888376737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:34.708019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-85f136b8-f38d-4559-9eac-6d6870dbc5d4 None None] MFA Rules not processed for user `cf40b4f00b3d41439c87967451e24661`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:34.708235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-fcbe3cd1-23fd-4052-b0c8-a8f888376737 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:34.709666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-fcbe3cd1-23fd-4052-b0c8-a8f888376737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:34.710734 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-85f136b8-f38d-4559-9eac-6d6870dbc5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:34.890116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 858/1694] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:34.890978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe3cd1-23fd-4052-b0c8-a8f888376737 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:34.891827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 837/1695] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:34 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:34.893658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55530c-0c93-4f2e-aa9f-5aedd37c12f1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:34.894130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55530c-0c93-4f2e-aa9f-5aedd37c12f1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:34.894421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55530c-0c93-4f2e-aa9f-5aedd37c12f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:34.924720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe3cd1-23fd-4052-b0c8-a8f888376737 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:34.937005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9f55530c-0c93-4f2e-aa9f-5aedd37c12f1 None None] MFA Rules not processed for user `ef098bbb9ee94bf4aadb4b59516890fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:34.940408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9f55530c-0c93-4f2e-aa9f-5aedd37c12f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:34.940915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae41d-0349-47fd-a349-45b0093c0f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:34.942137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae41d-0349-47fd-a349-45b0093c0f0c admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:34.942447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae41d-0349-47fd-a349-45b0093c0f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:34.942714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cae41d-0349-47fd-a349-45b0093c0f0c admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:34.943459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cae41d-0349-47fd-a349-45b0093c0f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:34.945460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cae41d-0349-47fd-a349-45b0093c0f0c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:35.017632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 859/1696] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:34 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:35.019192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae41d-0349-47fd-a349-45b0093c0f0c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:35.044186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cae41d-0349-47fd-a349-45b0093c0f0c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:35.057749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9b1fb-28e0-4657-b104-5421a3101d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:35.058366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9b1fb-28e0-4657-b104-5421a3101d48 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.058656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9b1fb-28e0-4657-b104-5421a3101d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.058939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9b1fb-28e0-4657-b104-5421a3101d48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.059724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9b1fb-28e0-4657-b104-5421a3101d48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:35.060040 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9b1fb-28e0-4657-b104-5421a3101d48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:35.064209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9b1fb-28e0-4657-b104-5421a3101d48 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:35.080858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 838/1697] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:35.082396 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55530c-0c93-4f2e-aa9f-5aedd37c12f1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:35.089916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9f55530c-0c93-4f2e-aa9f-5aedd37c12f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:35.112001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 860/1698] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:35.112906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9b1fb-28e0-4657-b104-5421a3101d48 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:35.141505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9b1fb-28e0-4657-b104-5421a3101d48 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:35.163577 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaa8eb-a074-427f-867f-3e26b9a4e466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:35.168051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaa8eb-a074-427f-867f-3e26b9a4e466 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.168388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaa8eb-a074-427f-867f-3e26b9a4e466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.168673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaa8eb-a074-427f-867f-3e26b9a4e466 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.169444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaa8eb-a074-427f-867f-3e26b9a4e466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:35.171098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaaa8eb-a074-427f-867f-3e26b9a4e466 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:35.220247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 861/1699] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:35.221109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaa8eb-a074-427f-867f-3e26b9a4e466 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:35.239840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaa8eb-a074-427f-867f-3e26b9a4e466 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:35.263763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55530c-0c93-4f2e-aa9f-5aedd37c12f1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:35.266924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438dcecf-6d72-4126-ba47-aff85c327170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:35.267753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dcecf-6d72-4126-ba47-aff85c327170 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.268051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dcecf-6d72-4126-ba47-aff85c327170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.268847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438dcecf-6d72-4126-ba47-aff85c327170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.269040 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438dcecf-6d72-4126-ba47-aff85c327170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:35.271538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438dcecf-6d72-4126-ba47-aff85c327170 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:35.277013 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16cb01-38ac-4ba7-a50e-80fc6a6943b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:35.278171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16cb01-38ac-4ba7-a50e-80fc6a6943b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.278468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16cb01-38ac-4ba7-a50e-80fc6a6943b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.278754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16cb01-38ac-4ba7-a50e-80fc6a6943b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.284591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:35.305237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16cb01-38ac-4ba7-a50e-80fc6a6943b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:35.306725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16cb01-38ac-4ba7-a50e-80fc6a6943b6 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:35.323444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 839/1700] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:35 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:35.324833 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322dd46-ea89-4818-9c29-1fbec3444797 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.325295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322dd46-ea89-4818-9c29-1fbec3444797 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.325538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322dd46-ea89-4818-9c29-1fbec3444797 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.327530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 840/1701] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jul 5 15:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:13:35.328389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322dd46-ea89-4818-9c29-1fbec3444797 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:35.350668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 862/1702] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:35 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:35.353833 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322dd46-ea89-4818-9c29-1fbec3444797 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:35.355386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df59c-3eab-45af-991b-da5fc11a1b13 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.355685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df59c-3eab-45af-991b-da5fc11a1b13 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.355968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5df59c-3eab-45af-991b-da5fc11a1b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.379771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:35.395401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-6a5df59c-3eab-45af-991b-da5fc11a1b13 None None] MFA Rules not processed for user `0187789947064682a8d9e685c2aadb9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:35.403401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6a5df59c-3eab-45af-991b-da5fc11a1b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:35.502261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a341c-a1ce-4db8-a0b7-69e4a5a183f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:35.503027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a341c-a1ce-4db8-a0b7-69e4a5a183f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.503346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a341c-a1ce-4db8-a0b7-69e4a5a183f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.503636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a341c-a1ce-4db8-a0b7-69e4a5a183f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.506680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a341c-a1ce-4db8-a0b7-69e4a5a183f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:35.509794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a341c-a1ce-4db8-a0b7-69e4a5a183f0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:35.542185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 841/1703] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:35.545805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625aff0e-5f5f-4ca4-b1fe-b43157846016 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.546567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625aff0e-5f5f-4ca4-b1fe-b43157846016 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.546850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625aff0e-5f5f-4ca4-b1fe-b43157846016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.590545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 863/1704] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:35.591626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5df59c-3eab-45af-991b-da5fc11a1b13 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:35.625026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5df59c-3eab-45af-991b-da5fc11a1b13 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:35.642184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89717b7b-52ae-41cf-9b56-32c221650077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:35.645554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89717b7b-52ae-41cf-9b56-32c221650077 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.645909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89717b7b-52ae-41cf-9b56-32c221650077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.646149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89717b7b-52ae-41cf-9b56-32c221650077 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.648035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89717b7b-52ae-41cf-9b56-32c221650077 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:35.652313 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89717b7b-52ae-41cf-9b56-32c221650077 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:35.662090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-625aff0e-5f5f-4ca4-b1fe-b43157846016 None None] MFA Rules not processed for user `cd7fa93466c54632ad95834297410e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:35.663279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-625aff0e-5f5f-4ca4-b1fe-b43157846016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:35.715136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 864/1705] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:35 2022] POST /identity/v3/users => generated 302 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:35.723893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89717b7b-52ae-41cf-9b56-32c221650077 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:35.730720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-89717b7b-52ae-41cf-9b56-32c221650077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:35.775181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89717b7b-52ae-41cf-9b56-32c221650077 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:35.794574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:35.795522 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.795808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.796093 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] PATH_INFO: `/v3/users/a2fc16b25b734f11a691a27fd35544e2` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.806859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2fc16b25b734f11a691a27fd35544e2)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:35.808893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:35.842035 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 842/1706] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:13:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:35.842913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625aff0e-5f5f-4ca4-b1fe-b43157846016 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:35.913241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625aff0e-5f5f-4ca4-b1fe-b43157846016 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:35.925489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024065d2-fc88-4680-aaab-c01d6497a440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:35.926996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024065d2-fc88-4680-aaab-c01d6497a440 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:35.927311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024065d2-fc88-4680-aaab-c01d6497a440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:35.927591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024065d2-fc88-4680-aaab-c01d6497a440 admin admin] PATH_INFO: `/v3/projects/c09e55ce38e347f08cb629c22ee5d16b/users/41a75990afdf40c0a76d7b499b9a35af/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:35.927689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2fc16b25b734f11a691a27fd35544e2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:35.928656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2fc16b25b734f11a691a27fd35544e2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:35.929735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2fc16b25b734f11a691a27fd35544e2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:35.941977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2fc16b25b734f11a691a27fd35544e2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:35.958060 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2fc16b25b734f11a691a27fd35544e2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:35.996995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024065d2-fc88-4680-aaab-c01d6497a440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c09e55ce38e347f08cb629c22ee5d16b, user_id=41a75990afdf40c0a76d7b499b9a35af, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:36.011843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024065d2-fc88-4680-aaab-c01d6497a440 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:36.025448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 865/1707] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:13:35 2022] DELETE /identity/v3/users/a2fc16b25b734f11a691a27fd35544e2 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:36.026793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:36.029558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:36.042214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 843/1708] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:35 2022] PUT /identity/v3/projects/c09e55ce38e347f08cb629c22ee5d16b/users/41a75990afdf40c0a76d7b499b9a35af/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:36.046949 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024065d2-fc88-4680-aaab-c01d6497a440 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:36.049615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-024065d2-fc88-4680-aaab-c01d6497a440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:36.144011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:36.195804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7319310f-66a0-4d51-829f-5e4d137ff419 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:36.218932 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024065d2-fc88-4680-aaab-c01d6497a440 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:36.230546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09797b7b-8428-4253-919e-bc84535a7ce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:36.231642 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09797b7b-8428-4253-919e-bc84535a7ce3 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.232242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09797b7b-8428-4253-919e-bc84535a7ce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.232821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09797b7b-8428-4253-919e-bc84535a7ce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.249910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-09797b7b-8428-4253-919e-bc84535a7ce3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:36.280553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09797b7b-8428-4253-919e-bc84535a7ce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:36.284863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09797b7b-8428-4253-919e-bc84535a7ce3 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:36.296929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 844/1709] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:36.298401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c3d4d-cbe5-46c8-a36f-efb05b000efa None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.298860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c3d4d-cbe5-46c8-a36f-efb05b000efa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.299780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c3d4d-cbe5-46c8-a36f-efb05b000efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.313661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230e5bc-006e-46a6-96e3-2f96376fd3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:36.321340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230e5bc-006e-46a6-96e3-2f96376fd3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.321867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230e5bc-006e-46a6-96e3-2f96376fd3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.322318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230e5bc-006e-46a6-96e3-2f96376fd3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.323817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230e5bc-006e-46a6-96e3-2f96376fd3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:36.332647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-073c3d4d-cbe5-46c8-a36f-efb05b000efa None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:36.336088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230e5bc-006e-46a6-96e3-2f96376fd3e8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:36.352607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-073c3d4d-cbe5-46c8-a36f-efb05b000efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:36.371091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 866/1710] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:36.372898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306423c-7af2-4f56-832a-14a21c4418ea None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.373227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306423c-7af2-4f56-832a-14a21c4418ea None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.373469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306423c-7af2-4f56-832a-14a21c4418ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.377883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 845/1711] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:36.382855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32040806-b038-481f-be10-b97bd7f4196a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.383243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32040806-b038-481f-be10-b97bd7f4196a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.383706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32040806-b038-481f-be10-b97bd7f4196a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.386800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 846/1712] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jul 5 15:13:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:13:36.389632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32040806-b038-481f-be10-b97bd7f4196a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:36.423665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32040806-b038-481f-be10-b97bd7f4196a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:36.438363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0306423c-7af2-4f56-832a-14a21c4418ea None None] MFA Rules not processed for user `41a75990afdf40c0a76d7b499b9a35af`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:36.439674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0306423c-7af2-4f56-832a-14a21c4418ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:36.440103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85711989-2bbc-4073-8fac-e5339444e378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:36.440866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85711989-2bbc-4073-8fac-e5339444e378 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.441223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85711989-2bbc-4073-8fac-e5339444e378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.441571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85711989-2bbc-4073-8fac-e5339444e378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.442932 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85711989-2bbc-4073-8fac-e5339444e378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:36.444967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85711989-2bbc-4073-8fac-e5339444e378 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:36.474385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 847/1713] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:36.478993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3c32e-eac8-4fb9-b02c-fc2168081ddd None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.480817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3c32e-eac8-4fb9-b02c-fc2168081ddd None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.480817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3c32e-eac8-4fb9-b02c-fc2168081ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.502062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:36.537198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-44f3c32e-eac8-4fb9-b02c-fc2168081ddd None None] MFA Rules not processed for user `cd7fa93466c54632ad95834297410e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:36.542518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-44f3c32e-eac8-4fb9-b02c-fc2168081ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:36.565148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 848/1714] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:13:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:36.565890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3c32e-eac8-4fb9-b02c-fc2168081ddd None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:36.575635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3c32e-eac8-4fb9-b02c-fc2168081ddd None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:36.580078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155a44d-6882-4b88-84eb-9d446b5e474c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:36.581106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155a44d-6882-4b88-84eb-9d446b5e474c service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.581106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155a44d-6882-4b88-84eb-9d446b5e474c service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.581406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155a44d-6882-4b88-84eb-9d446b5e474c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.595026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155a44d-6882-4b88-84eb-9d446b5e474c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:36.596493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a155a44d-6882-4b88-84eb-9d446b5e474c service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:36.598047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 867/1715] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:36.598867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306423c-7af2-4f56-832a-14a21c4418ea None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:36.618186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306423c-7af2-4f56-832a-14a21c4418ea None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:36.622619 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 849/1716] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:36.625618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155a44d-6882-4b88-84eb-9d446b5e474c service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:36.628018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a155a44d-6882-4b88-84eb-9d446b5e474c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:36.631090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f3447-5b1e-4f23-8eb8-cb057fac2c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:36.636101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f3447-5b1e-4f23-8eb8-cb057fac2c3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.636816 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f3447-5b1e-4f23-8eb8-cb057fac2c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.637626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f3447-5b1e-4f23-8eb8-cb057fac2c3b admin admin] PATH_INFO: `/v3/projects/2f266029fd86485f970d82b4a68de060/users/b1bb56a8ecce4608a269b7714fdd9adf/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.667030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a155a44d-6882-4b88-84eb-9d446b5e474c service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:36.674895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac2bbb-6254-4b7f-b028-b80565f23888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:36.675769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac2bbb-6254-4b7f-b028-b80565f23888 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.676599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac2bbb-6254-4b7f-b028-b80565f23888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.676912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac2bbb-6254-4b7f-b028-b80565f23888 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.677701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac2bbb-6254-4b7f-b028-b80565f23888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:36.678778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f3447-5b1e-4f23-8eb8-cb057fac2c3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f266029fd86485f970d82b4a68de060, user_id=b1bb56a8ecce4608a269b7714fdd9adf, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:36.679663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac2bbb-6254-4b7f-b028-b80565f23888 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:36.683001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f3447-5b1e-4f23-8eb8-cb057fac2c3b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:36.714362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 868/1717] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:36 2022] PUT /identity/v3/projects/2f266029fd86485f970d82b4a68de060/users/b1bb56a8ecce4608a269b7714fdd9adf/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:36.715820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f3447-5b1e-4f23-8eb8-cb057fac2c3b admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:36.767852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 850/1718] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:36 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:36.769368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c49d6d-d284-4335-81ca-75359919b0a0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:36.772283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c49d6d-d284-4335-81ca-75359919b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:36.772804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c49d6d-d284-4335-81ca-75359919b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:36.816869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d6c49d6d-d284-4335-81ca-75359919b0a0 None None] MFA Rules not processed for user `8bbc454737084513baa37072cd160e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:36.821422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d6c49d6d-d284-4335-81ca-75359919b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:36.825360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f3447-5b1e-4f23-8eb8-cb057fac2c3b admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:36.843838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:37.019498 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14853b-184c-421d-8fe8-12fb1e912858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:37.020262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14853b-184c-421d-8fe8-12fb1e912858 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.020626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14853b-184c-421d-8fe8-12fb1e912858 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.021074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14853b-184c-421d-8fe8-12fb1e912858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.025501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 851/1719] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:37.028120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40bf4-ad05-4cc7-bb69-6516c0b9131e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.028576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40bf4-ad05-4cc7-bb69-6516c0b9131e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.029383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40bf4-ad05-4cc7-bb69-6516c0b9131e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.030389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14853b-184c-421d-8fe8-12fb1e912858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:37.032026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14853b-184c-421d-8fe8-12fb1e912858 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:37.044486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 869/1720] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:37.045320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14853b-184c-421d-8fe8-12fb1e912858 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.082128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c7b40bf4-ad05-4cc7-bb69-6516c0b9131e None None] MFA Rules not processed for user `b1bb56a8ecce4608a269b7714fdd9adf`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:37.083446 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c7b40bf4-ad05-4cc7-bb69-6516c0b9131e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:37.142459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14853b-184c-421d-8fe8-12fb1e912858 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:37.150181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19524ee-1baf-48b4-a22e-1f90f146771d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:37.151839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19524ee-1baf-48b4-a22e-1f90f146771d admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.152281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19524ee-1baf-48b4-a22e-1f90f146771d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.152691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19524ee-1baf-48b4-a22e-1f90f146771d admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.153594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19524ee-1baf-48b4-a22e-1f90f146771d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:37.157381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19524ee-1baf-48b4-a22e-1f90f146771d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:37.187284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 852/1721] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:37.188377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b40bf4-ad05-4cc7-bb69-6516c0b9131e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.195834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c7b40bf4-ad05-4cc7-bb69-6516c0b9131e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:37.195931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 870/1722] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:37 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:37.206785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19524ee-1baf-48b4-a22e-1f90f146771d admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.222450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19524ee-1baf-48b4-a22e-1f90f146771d admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:37.234325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfae8d2-80d6-4acb-8a70-bc260108bd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:37.235108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfae8d2-80d6-4acb-8a70-bc260108bd95 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.235995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfae8d2-80d6-4acb-8a70-bc260108bd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.236506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfae8d2-80d6-4acb-8a70-bc260108bd95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.237350 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfae8d2-80d6-4acb-8a70-bc260108bd95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:37.239589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfae8d2-80d6-4acb-8a70-bc260108bd95 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:37.287739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 871/1723] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:37 2022] POST /identity/v3/projects => generated 405 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:37.288728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfae8d2-80d6-4acb-8a70-bc260108bd95 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.313126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:37.353190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b40bf4-ad05-4cc7-bb69-6516c0b9131e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:37.436650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfae8d2-80d6-4acb-8a70-bc260108bd95 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:37.449031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1ac47-67ac-4dfc-a90c-9d256dc132bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:37.449916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1ac47-67ac-4dfc-a90c-9d256dc132bb admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.450254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1ac47-67ac-4dfc-a90c-9d256dc132bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.450541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1ac47-67ac-4dfc-a90c-9d256dc132bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.462828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1ac47-67ac-4dfc-a90c-9d256dc132bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:37.466416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1ac47-67ac-4dfc-a90c-9d256dc132bb admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:37.467906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669024d-d57d-4795-98e8-47470aae75b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:37.470235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669024d-d57d-4795-98e8-47470aae75b1 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.471447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669024d-d57d-4795-98e8-47470aae75b1 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.471891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669024d-d57d-4795-98e8-47470aae75b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.475891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c669024d-d57d-4795-98e8-47470aae75b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:37.497522 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 872/1724] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:37.498296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1ac47-67ac-4dfc-a90c-9d256dc132bb admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.506006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669024d-d57d-4795-98e8-47470aae75b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:37.507984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669024d-d57d-4795-98e8-47470aae75b1 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:37.522373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1ac47-67ac-4dfc-a90c-9d256dc132bb admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:37.534420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 853/1725] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:37.535319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669024d-d57d-4795-98e8-47470aae75b1 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.539189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26aa332-d3d5-4489-a8e9-f901d6f9f556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:37.539886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aa332-d3d5-4489-a8e9-f901d6f9f556 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.540213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aa332-d3d5-4489-a8e9-f901d6f9f556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.540462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26aa332-d3d5-4489-a8e9-f901d6f9f556 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.541174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26aa332-d3d5-4489-a8e9-f901d6f9f556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:37.547194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26aa332-d3d5-4489-a8e9-f901d6f9f556 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:37.551424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669024d-d57d-4795-98e8-47470aae75b1 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:37.563105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648193b-e82f-4d9c-8782-1a9080908a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:37.568613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648193b-e82f-4d9c-8782-1a9080908a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.569182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648193b-e82f-4d9c-8782-1a9080908a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.570045 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648193b-e82f-4d9c-8782-1a9080908a9e admin admin] PATH_INFO: `/v3/projects/72fd939407ce4ef9b35cb9b9a8a6d974/users/bb58af1a40e64ed7ba6171751a955538/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.570774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:37.607671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 873/1726] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:37 2022] POST /identity/v3/users => generated 309 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:37.612392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26aa332-d3d5-4489-a8e9-f901d6f9f556 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.627459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648193b-e82f-4d9c-8782-1a9080908a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72fd939407ce4ef9b35cb9b9a8a6d974, user_id=bb58af1a40e64ed7ba6171751a955538, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:37.637789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26aa332-d3d5-4489-a8e9-f901d6f9f556 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:37.641970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648193b-e82f-4d9c-8782-1a9080908a9e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:37.644273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f3bfb-0159-4883-9e2f-aead631d1fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:37.645959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f3bfb-0159-4883-9e2f-aead631d1fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.646389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f3bfb-0159-4883-9e2f-aead631d1fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.646817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f3bfb-0159-4883-9e2f-aead631d1fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.647750 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f3bfb-0159-4883-9e2f-aead631d1fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:37.650280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219f3bfb-0159-4883-9e2f-aead631d1fb8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:37.673399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 854/1727] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:37 2022] PUT /identity/v3/projects/72fd939407ce4ef9b35cb9b9a8a6d974/users/bb58af1a40e64ed7ba6171751a955538/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:37.678702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648193b-e82f-4d9c-8782-1a9080908a9e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.701155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 874/1728] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:13:37.701968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f3bfb-0159-4883-9e2f-aead631d1fb8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.732847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f3bfb-0159-4883-9e2f-aead631d1fb8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:37.865607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648193b-e82f-4d9c-8782-1a9080908a9e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:37.869875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391998b5-c3bd-49aa-a121-e9e91ad98cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:37.870916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391998b5-c3bd-49aa-a121-e9e91ad98cb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.871238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391998b5-c3bd-49aa-a121-e9e91ad98cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.871521 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391998b5-c3bd-49aa-a121-e9e91ad98cb3 admin admin] PATH_INFO: `/v3/projects/2a1dd854625b4334bf5130fea9877a3d/users/2660d757582146f59eeaf493c55f8e79/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.880584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fbcde-86f8-4dcb-ab93-fb438584dec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:37.881237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fbcde-86f8-4dcb-ab93-fb438584dec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:37.881561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fbcde-86f8-4dcb-ab93-fb438584dec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:37.881796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fbcde-86f8-4dcb-ab93-fb438584dec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:37.882600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fbcde-86f8-4dcb-ab93-fb438584dec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:37.885376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fbcde-86f8-4dcb-ab93-fb438584dec1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:37.913084 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 855/1729] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:37.916070 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fbcde-86f8-4dcb-ab93-fb438584dec1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.916276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391998b5-c3bd-49aa-a121-e9e91ad98cb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1dd854625b4334bf5130fea9877a3d, user_id=2660d757582146f59eeaf493c55f8e79, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:37.921459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-866fbcde-86f8-4dcb-ab93-fb438584dec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:37.926459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391998b5-c3bd-49aa-a121-e9e91ad98cb3 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:37.943656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fbcde-86f8-4dcb-ab93-fb438584dec1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:37.956230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 875/1730] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:37 2022] PUT /identity/v3/projects/2a1dd854625b4334bf5130fea9877a3d/users/2660d757582146f59eeaf493c55f8e79/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:37.958042 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391998b5-c3bd-49aa-a121-e9e91ad98cb3 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:37.960729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-391998b5-c3bd-49aa-a121-e9e91ad98cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:38.061101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391998b5-c3bd-49aa-a121-e9e91ad98cb3 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:38.071615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:38.073537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:38.074072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:38.074556 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] PATH_INFO: `/v3/users/eadc03a40546486585def15c075da201` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:38.076629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eadc03a40546486585def15c075da201)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:38.079260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:38.082589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:38.083558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:38.083807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:38.084083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] PATH_INFO: `/v3/projects/9b1d323459494d6fa4b206d9aeb403db` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:38.085514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b1d323459494d6fa4b206d9aeb403db)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:38.087931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:38.092522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:38.170879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eadc03a40546486585def15c075da201'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:38.173975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eadc03a40546486585def15c075da201'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:38.175419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eadc03a40546486585def15c075da201'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:38.183101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b1d323459494d6fa4b206d9aeb403db', 'request_id': 'req-0d37e035-10e1-47c1-8801-347c8f929f79'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:38.187551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eadc03a40546486585def15c075da201'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:38.199871 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b1d323459494d6fa4b206d9aeb403db', 'request_id': 'req-0d37e035-10e1-47c1-8801-347c8f929f79'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:38.217267 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eadc03a40546486585def15c075da201'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:38.222288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b1d323459494d6fa4b206d9aeb403db', 'request_id': 'req-0d37e035-10e1-47c1-8801-347c8f929f79'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:38.257370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:38.267697 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] The token cache is being invalidate because project 9b1d323459494d6fa4b206d9aeb403db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:13:38.268041 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:38.272325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:38.281699 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 856/1731] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:13:37 2022] DELETE /identity/v3/projects/9b1d323459494d6fa4b206d9aeb403db => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:38.283715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:38.288832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:38.300443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 876/1732] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:13:37 2022] DELETE /identity/v3/users/eadc03a40546486585def15c075da201 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:38.301910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:38.305115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:38.471988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78188611-a2bb-47b1-b208-434dc3c38f66 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:38.479850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da7302-02c6-44e8-b276-1e5af2a8f684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:38.480867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da7302-02c6-44e8-b276-1e5af2a8f684 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:38.481494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da7302-02c6-44e8-b276-1e5af2a8f684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:38.482110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da7302-02c6-44e8-b276-1e5af2a8f684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:38.483247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da7302-02c6-44e8-b276-1e5af2a8f684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:38.487778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da7302-02c6-44e8-b276-1e5af2a8f684 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:38.489913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d37e035-10e1-47c1-8801-347c8f929f79 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:38.507376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6075d-b9fa-4f0f-946b-503d7edd9e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:38.508380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6075d-b9fa-4f0f-946b-503d7edd9e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:38.508666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6075d-b9fa-4f0f-946b-503d7edd9e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:38.508945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6075d-b9fa-4f0f-946b-503d7edd9e48 admin admin] PATH_INFO: `/v3/projects/72fd939407ce4ef9b35cb9b9a8a6d974/users/bb58af1a40e64ed7ba6171751a955538/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:38.515604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6075d-b9fa-4f0f-946b-503d7edd9e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72fd939407ce4ef9b35cb9b9a8a6d974, user_id=bb58af1a40e64ed7ba6171751a955538, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:38.518648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6075d-b9fa-4f0f-946b-503d7edd9e48 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:38.525520 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 877/1733] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:38.527177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658690c-9f52-4ffe-a54c-b22bf5a5e062 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:38.527889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658690c-9f52-4ffe-a54c-b22bf5a5e062 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:38.528305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8658690c-9f52-4ffe-a54c-b22bf5a5e062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:38.560078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 857/1734] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:38 2022] PUT /identity/v3/projects/72fd939407ce4ef9b35cb9b9a8a6d974/users/bb58af1a40e64ed7ba6171751a955538/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:38.569991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ee46-da02-4d09-b147-d004def9fc2d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:38.570293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ee46-da02-4d09-b147-d004def9fc2d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:38.573773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-8658690c-9f52-4ffe-a54c-b22bf5a5e062 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:38.575557 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842ee46-da02-4d09-b147-d004def9fc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:38.582582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8658690c-9f52-4ffe-a54c-b22bf5a5e062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:38.619274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4842ee46-da02-4d09-b147-d004def9fc2d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:38.620534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4842ee46-da02-4d09-b147-d004def9fc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:38.710947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 878/1735] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:38.713688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658690c-9f52-4ffe-a54c-b22bf5a5e062 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:38.716248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8658690c-9f52-4ffe-a54c-b22bf5a5e062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:38.756615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8658690c-9f52-4ffe-a54c-b22bf5a5e062 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:38.795034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 858/1736] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:38.800382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c7965-70a9-4566-b907-817364572d00 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:38.804049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d7f6e-ad44-4427-a96e-b244d67ea64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:38.804418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c7965-70a9-4566-b907-817364572d00 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:38.804750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c7965-70a9-4566-b907-817364572d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:38.806662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7f6e-ad44-4427-a96e-b244d67ea64d admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:38.806894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7f6e-ad44-4427-a96e-b244d67ea64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:38.807236 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d7f6e-ad44-4427-a96e-b244d67ea64d admin admin] PATH_INFO: `/v3/projects/2a1dd854625b4334bf5130fea9877a3d/users/2660d757582146f59eeaf493c55f8e79/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:38.815971 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d7f6e-ad44-4427-a96e-b244d67ea64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1dd854625b4334bf5130fea9877a3d, user_id=2660d757582146f59eeaf493c55f8e79, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:38.830374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d7f6e-ad44-4427-a96e-b244d67ea64d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:38.858327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 879/1737] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:38 2022] PUT /identity/v3/projects/2a1dd854625b4334bf5130fea9877a3d/users/2660d757582146f59eeaf493c55f8e79/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:38.863165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f426ddc-39e7-4d7f-b83e-7664572ff973 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:38.863165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f426ddc-39e7-4d7f-b83e-7664572ff973 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:38.864153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f426ddc-39e7-4d7f-b83e-7664572ff973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:38.881983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-648c7965-70a9-4566-b907-817364572d00 None None] MFA Rules not processed for user `bb58af1a40e64ed7ba6171751a955538`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:38.883531 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-648c7965-70a9-4566-b907-817364572d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:38.911198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-2f426ddc-39e7-4d7f-b83e-7664572ff973 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:38.915876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2f426ddc-39e7-4d7f-b83e-7664572ff973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:38.916635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:39.039477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 880/1738] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:39.041005 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f426ddc-39e7-4d7f-b83e-7664572ff973 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:39.042702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2f426ddc-39e7-4d7f-b83e-7664572ff973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:39.090372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 859/1739] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:39.090372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d82684-8349-404b-add5-647044fa6986 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:39.090372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d82684-8349-404b-add5-647044fa6986 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:39.090372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d82684-8349-404b-add5-647044fa6986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:39.127969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:39.139077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-24d82684-8349-404b-add5-647044fa6986 None None] MFA Rules not processed for user `2660d757582146f59eeaf493c55f8e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:39.143421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-24d82684-8349-404b-add5-647044fa6986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:39.298985 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f426ddc-39e7-4d7f-b83e-7664572ff973 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:39.299870 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 860/1740] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:39 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:39.301801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d82684-8349-404b-add5-647044fa6986 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:39.311418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96518ad2-c9d3-418f-b61d-00a5d7ba79ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:39.314630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96518ad2-c9d3-418f-b61d-00a5d7ba79ac service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:39.314897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96518ad2-c9d3-418f-b61d-00a5d7ba79ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:39.316249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96518ad2-c9d3-418f-b61d-00a5d7ba79ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:39.337544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96518ad2-c9d3-418f-b61d-00a5d7ba79ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:39.339676 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96518ad2-c9d3-418f-b61d-00a5d7ba79ac service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:39.349317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d82684-8349-404b-add5-647044fa6986 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:39.367190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fccc2-52cf-4ebe-b8b8-becdae3be743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:39.369919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fccc2-52cf-4ebe-b8b8-becdae3be743 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:39.371491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fccc2-52cf-4ebe-b8b8-becdae3be743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:39.371800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fccc2-52cf-4ebe-b8b8-becdae3be743 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:39.373165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 881/1741] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:39.373854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96518ad2-c9d3-418f-b61d-00a5d7ba79ac service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:39.377620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fccc2-52cf-4ebe-b8b8-becdae3be743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:39.378089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fccc2-52cf-4ebe-b8b8-becdae3be743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:39.380445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-96518ad2-c9d3-418f-b61d-00a5d7ba79ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:39.383836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966fccc2-52cf-4ebe-b8b8-becdae3be743 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:39.444300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 861/1742] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:39.445701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fccc2-52cf-4ebe-b8b8-becdae3be743 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:39.447563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-966fccc2-52cf-4ebe-b8b8-becdae3be743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:39.596715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fccc2-52cf-4ebe-b8b8-becdae3be743 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:39.605261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48e320-59a1-4065-849e-93a8b57cd237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:39.607243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48e320-59a1-4065-849e-93a8b57cd237 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:39.607741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48e320-59a1-4065-849e-93a8b57cd237 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:39.609660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48e320-59a1-4065-849e-93a8b57cd237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:39.623469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96518ad2-c9d3-418f-b61d-00a5d7ba79ac service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:39.628656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebfd41-98b8-4990-bab3-fa4c7ddf7a42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:39.629226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48e320-59a1-4065-849e-93a8b57cd237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:39.629353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebfd41-98b8-4990-bab3-fa4c7ddf7a42 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:39.629679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebfd41-98b8-4990-bab3-fa4c7ddf7a42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:39.630010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebfd41-98b8-4990-bab3-fa4c7ddf7a42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:39.634290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48e320-59a1-4065-849e-93a8b57cd237 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:39.656445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ebfd41-98b8-4990-bab3-fa4c7ddf7a42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:39.657944 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ebfd41-98b8-4990-bab3-fa4c7ddf7a42 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:39.678485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 862/1743] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:39 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:39.680307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48e320-59a1-4065-849e-93a8b57cd237 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:39.690701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 882/1744] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:39 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:39.709118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48e320-59a1-4065-849e-93a8b57cd237 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:39.745522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48037eb-5cd7-48fa-a25f-661c5d8495c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:39.749724 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48037eb-5cd7-48fa-a25f-661c5d8495c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:39.752205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48037eb-5cd7-48fa-a25f-661c5d8495c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:39.752723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48037eb-5cd7-48fa-a25f-661c5d8495c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:39.754858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48037eb-5cd7-48fa-a25f-661c5d8495c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:39.758853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48037eb-5cd7-48fa-a25f-661c5d8495c3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:39.848652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 863/1745] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:39 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:39.849950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebfd41-98b8-4990-bab3-fa4c7ddf7a42 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:39.870448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebfd41-98b8-4990-bab3-fa4c7ddf7a42 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:39.889264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48037eb-5cd7-48fa-a25f-661c5d8495c3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:39.896743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:40.026212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48037eb-5cd7-48fa-a25f-661c5d8495c3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:40.037763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac4c55-1c10-4a95-ba7e-1e67fba3dbb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:40.038692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac4c55-1c10-4a95-ba7e-1e67fba3dbb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:40.039328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac4c55-1c10-4a95-ba7e-1e67fba3dbb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:40.039835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac4c55-1c10-4a95-ba7e-1e67fba3dbb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:40.048355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e8ac4c55-1c10-4a95-ba7e-1e67fba3dbb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:40.056824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51992790-e895-40b7-896d-37f3151e9bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:40.057863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992790-e895-40b7-896d-37f3151e9bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:40.058158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992790-e895-40b7-896d-37f3151e9bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:40.058445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51992790-e895-40b7-896d-37f3151e9bf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:40.059319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51992790-e895-40b7-896d-37f3151e9bf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:40.063333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51992790-e895-40b7-896d-37f3151e9bf9 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:40.092980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 883/1746] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:39 2022] POST /identity/v3/users => generated 294 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:40.093974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51992790-e895-40b7-896d-37f3151e9bf9 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:40.117751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51992790-e895-40b7-896d-37f3151e9bf9 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:40.134611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:40.136165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:40.136675 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:40.137169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] PATH_INFO: `/v3/users/b1bb56a8ecce4608a269b7714fdd9adf` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:40.139750 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1bb56a8ecce4608a269b7714fdd9adf)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:40.142650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:40.222608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac4c55-1c10-4a95-ba7e-1e67fba3dbb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:40.228508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac4c55-1c10-4a95-ba7e-1e67fba3dbb5 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:40.252380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 864/1747] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:13:40.254597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac4c55-1c10-4a95-ba7e-1e67fba3dbb5 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:40.273777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:40.277224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac4c55-1c10-4a95-ba7e-1e67fba3dbb5 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:40.281003 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1bb56a8ecce4608a269b7714fdd9adf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:40.281965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1bb56a8ecce4608a269b7714fdd9adf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:40.285828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1bb56a8ecce4608a269b7714fdd9adf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:40.286843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f816f0-f9a7-47e8-ae7d-3f0d2acce9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:40.287781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f816f0-f9a7-47e8-ae7d-3f0d2acce9eb admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:40.288390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f816f0-f9a7-47e8-ae7d-3f0d2acce9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:40.288783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f816f0-f9a7-47e8-ae7d-3f0d2acce9eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:40.289695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f816f0-f9a7-47e8-ae7d-3f0d2acce9eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:40.292672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f816f0-f9a7-47e8-ae7d-3f0d2acce9eb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:40.307969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 865/1748] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:40.313050 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f816f0-f9a7-47e8-ae7d-3f0d2acce9eb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:40.320473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-47f816f0-f9a7-47e8-ae7d-3f0d2acce9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:40.325403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1bb56a8ecce4608a269b7714fdd9adf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:40.334743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f816f0-f9a7-47e8-ae7d-3f0d2acce9eb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:40.346512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcdd25-9552-4ced-ba5d-c323869a86f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:40.348347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcdd25-9552-4ced-ba5d-c323869a86f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:40.350067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcdd25-9552-4ced-ba5d-c323869a86f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:40.350368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcdd25-9552-4ced-ba5d-c323869a86f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:40.351656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcdd25-9552-4ced-ba5d-c323869a86f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:40.355897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85f63902-4a4b-4fdf-bc20-8c4d165d73a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1bb56a8ecce4608a269b7714fdd9adf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:40.360039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcdd25-9552-4ced-ba5d-c323869a86f7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:40.385128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 866/1749] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:40.386174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcdd25-9552-4ced-ba5d-c323869a86f7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:40.387774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 884/1750] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:13:40 2022] DELETE /identity/v3/users/b1bb56a8ecce4608a269b7714fdd9adf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:40.392480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bceca8-691e-47a6-b2b4-d0f2a8241cec None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:40.392908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bceca8-691e-47a6-b2b4-d0f2a8241cec None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:40.393304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bceca8-691e-47a6-b2b4-d0f2a8241cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:40.417975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcdd25-9552-4ced-ba5d-c323869a86f7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:40.432243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-30bceca8-691e-47a6-b2b4-d0f2a8241cec None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:40.440293 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-30bceca8-691e-47a6-b2b4-d0f2a8241cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:40.573193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e0afc-af5e-49a4-a6ce-6af97ed9eecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:40.574446 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e0afc-af5e-49a4-a6ce-6af97ed9eecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:40.574889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e0afc-af5e-49a4-a6ce-6af97ed9eecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:40.575421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e0afc-af5e-49a4-a6ce-6af97ed9eecd admin admin] PATH_INFO: `/v3/projects/98eff08c2639423cb6b73ee9035e3818/users/625f75bf4b604d79ab3af239da62fbbc/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:40.608656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e0afc-af5e-49a4-a6ce-6af97ed9eecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=98eff08c2639423cb6b73ee9035e3818, user_id=625f75bf4b604d79ab3af239da62fbbc, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:40.611627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e0afc-af5e-49a4-a6ce-6af97ed9eecd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:40.655376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 867/1751] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:40 2022] PUT /identity/v3/projects/98eff08c2639423cb6b73ee9035e3818/users/625f75bf4b604d79ab3af239da62fbbc/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:40.668576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada8ab2-7682-440d-a8d7-a51317f106ea None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:40.668937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada8ab2-7682-440d-a8d7-a51317f106ea None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:40.669139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada8ab2-7682-440d-a8d7-a51317f106ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:40.710265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2ada8ab2-7682-440d-a8d7-a51317f106ea None None] MFA Rules not processed for user `625f75bf4b604d79ab3af239da62fbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:40.716280 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2ada8ab2-7682-440d-a8d7-a51317f106ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:40.724087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 885/1752] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:13:40.734109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bceca8-691e-47a6-b2b4-d0f2a8241cec None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:40.735952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-30bceca8-691e-47a6-b2b4-d0f2a8241cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:40.743118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:40.893883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 868/1753] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:40.895527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada8ab2-7682-440d-a8d7-a51317f106ea None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:40.917829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bceca8-691e-47a6-b2b4-d0f2a8241cec None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:41.029735 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada8ab2-7682-440d-a8d7-a51317f106ea None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:41.030834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db75f60-56c3-44af-ad2b-2595cc6cf1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:41.032071 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db75f60-56c3-44af-ad2b-2595cc6cf1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:41.032470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db75f60-56c3-44af-ad2b-2595cc6cf1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:41.032878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db75f60-56c3-44af-ad2b-2595cc6cf1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:41.052881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be164a46-1082-4cc0-8bf1-46b9bf8457d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:41.053892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be164a46-1082-4cc0-8bf1-46b9bf8457d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:41.054504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be164a46-1082-4cc0-8bf1-46b9bf8457d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:41.056981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be164a46-1082-4cc0-8bf1-46b9bf8457d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:41.056981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be164a46-1082-4cc0-8bf1-46b9bf8457d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:41.058740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be164a46-1082-4cc0-8bf1-46b9bf8457d7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:41.062958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db75f60-56c3-44af-ad2b-2595cc6cf1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:41.064556 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db75f60-56c3-44af-ad2b-2595cc6cf1f1 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:41.077635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 886/1754] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:13:41.079290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db75f60-56c3-44af-ad2b-2595cc6cf1f1 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:41.084955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0db75f60-56c3-44af-ad2b-2595cc6cf1f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:41.108237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db75f60-56c3-44af-ad2b-2595cc6cf1f1 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:41.115689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:41.116682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:41.117739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:41.118044 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] PATH_INFO: `/v3/projects/c08409687cb64fa4971e1bae473c9230` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:41.119455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c08409687cb64fa4971e1bae473c9230)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:41.121190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: WARNING py.warnings [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 05 15:13:41.121190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: warnings.warn(msg) Jul 05 15:13:41.121190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: Jul 05 15:13:41.124745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:41.152399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:41.168439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 869/1755] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:40 2022] POST /identity/v3/projects => generated 387 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:41.170862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be164a46-1082-4cc0-8bf1-46b9bf8457d7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:41.231215 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c08409687cb64fa4971e1bae473c9230', 'request_id': 'req-a4327f00-c820-4536-90e8-f8df68367f86'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:41.243722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c08409687cb64fa4971e1bae473c9230', 'request_id': 'req-a4327f00-c820-4536-90e8-f8df68367f86'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:41.263608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c08409687cb64fa4971e1bae473c9230', 'request_id': 'req-a4327f00-c820-4536-90e8-f8df68367f86'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:41.278351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] The token cache is being invalidate because project c08409687cb64fa4971e1bae473c9230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:13:41.278949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:41.279666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:41.281492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 887/1756] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:13:41 2022] DELETE /identity/v3/projects/c08409687cb64fa4971e1bae473c9230 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:41.285933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:41.287387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:41.359863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be164a46-1082-4cc0-8bf1-46b9bf8457d7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:41.384652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb4116-15ec-42c1-9c34-2ffbde4f0573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:41.385513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb4116-15ec-42c1-9c34-2ffbde4f0573 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:41.385797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb4116-15ec-42c1-9c34-2ffbde4f0573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:41.386085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb4116-15ec-42c1-9c34-2ffbde4f0573 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:41.386809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb4116-15ec-42c1-9c34-2ffbde4f0573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:41.387075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4327f00-c820-4536-90e8-f8df68367f86 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:41.388805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb4116-15ec-42c1-9c34-2ffbde4f0573 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:41.393838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:41.395410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:41.395898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:41.397608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] PATH_INFO: `/v3/users/cf40b4f00b3d41439c87967451e24661` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:41.399633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf40b4f00b3d41439c87967451e24661)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:41.402170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:41.435036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 870/1757] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:41 2022] POST /identity/v3/users => generated 294 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:41.440881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb4116-15ec-42c1-9c34-2ffbde4f0573 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:41.458252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-29eb4116-15ec-42c1-9c34-2ffbde4f0573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:41.480301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf40b4f00b3d41439c87967451e24661'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:41.481442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf40b4f00b3d41439c87967451e24661'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:41.483311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf40b4f00b3d41439c87967451e24661'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:41.493238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf40b4f00b3d41439c87967451e24661'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:41.508306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf40b4f00b3d41439c87967451e24661'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:41.517006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb4116-15ec-42c1-9c34-2ffbde4f0573 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:41.533674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65803d8d-2743-4fec-92b3-98e2fda9101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:41.534374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65803d8d-2743-4fec-92b3-98e2fda9101f admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:41.534664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65803d8d-2743-4fec-92b3-98e2fda9101f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:41.535204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65803d8d-2743-4fec-92b3-98e2fda9101f admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:41.539892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65803d8d-2743-4fec-92b3-98e2fda9101f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:41.542998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65803d8d-2743-4fec-92b3-98e2fda9101f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:41.590169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 871/1758] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:41.595452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65803d8d-2743-4fec-92b3-98e2fda9101f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:41.602733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-65803d8d-2743-4fec-92b3-98e2fda9101f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:41.621907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 888/1759] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:13:41 2022] DELETE /identity/v3/users/cf40b4f00b3d41439c87967451e24661 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:41.625716 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65803d8d-2743-4fec-92b3-98e2fda9101f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:41.640065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:41.782257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b8871-cd44-45a9-9e83-b2aff6b02326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:41.787511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b8871-cd44-45a9-9e83-b2aff6b02326 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:41.788498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b8871-cd44-45a9-9e83-b2aff6b02326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:41.789931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b8871-cd44-45a9-9e83-b2aff6b02326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:41.793207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b8871-cd44-45a9-9e83-b2aff6b02326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:41.797657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b8871-cd44-45a9-9e83-b2aff6b02326 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:41.849111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 872/1760] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:41.851180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:41.879075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512902-2e88-4688-9c27-32e9b644babd admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:41.894084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d61103-a35a-4c63-90d9-ea74d68f8034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:41.948766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d61103-a35a-4c63-90d9-ea74d68f8034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:41.949406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d61103-a35a-4c63-90d9-ea74d68f8034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:41.951647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d61103-a35a-4c63-90d9-ea74d68f8034 admin admin] PATH_INFO: `/v3/projects/387c2b2715af4903bab3420b36e54681/users/687f98967c844f5da958dea2fae17671/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:42.023250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d61103-a35a-4c63-90d9-ea74d68f8034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387c2b2715af4903bab3420b36e54681, user_id=687f98967c844f5da958dea2fae17671, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:42.032899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d61103-a35a-4c63-90d9-ea74d68f8034 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:42.049963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:42.062871 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 889/1761] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:41 2022] PUT /identity/v3/projects/387c2b2715af4903bab3420b36e54681/users/687f98967c844f5da958dea2fae17671/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:42.072852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc9e40-d60d-4e15-b396-41163d6ce23f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:42.073148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc9e40-d60d-4e15-b396-41163d6ce23f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:42.073979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc9e40-d60d-4e15-b396-41163d6ce23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:42.080802 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e0083-1429-40e0-8060-a35fa606c43c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:42.081090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e0083-1429-40e0-8060-a35fa606c43c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:42.081377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7e0083-1429-40e0-8060-a35fa606c43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:42.152904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-23cc9e40-d60d-4e15-b396-41163d6ce23f None None] MFA Rules not processed for user `687f98967c844f5da958dea2fae17671`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:42.154187 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-23cc9e40-d60d-4e15-b396-41163d6ce23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:42.157890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-fd7e0083-1429-40e0-8060-a35fa606c43c None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:42.160660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-fd7e0083-1429-40e0-8060-a35fa606c43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:42.393853 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 890/1762] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:13:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:13:42.459650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 873/1763] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:42.464532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e0083-1429-40e0-8060-a35fa606c43c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:42.628346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7e0083-1429-40e0-8060-a35fa606c43c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:42.638030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5171a3-2659-4482-94d8-fccad2039836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:42.640640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5171a3-2659-4482-94d8-fccad2039836 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:42.640640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5171a3-2659-4482-94d8-fccad2039836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:42.640924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5171a3-2659-4482-94d8-fccad2039836 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:42.649825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc9e40-d60d-4e15-b396-41163d6ce23f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:42.652762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-23cc9e40-d60d-4e15-b396-41163d6ce23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:42.687498 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5171a3-2659-4482-94d8-fccad2039836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:42.699963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5171a3-2659-4482-94d8-fccad2039836 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:42.702112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc9e40-d60d-4e15-b396-41163d6ce23f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:42.715955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:42.717300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:42.718239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:42.719082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] PATH_INFO: `/v3/projects/2f266029fd86485f970d82b4a68de060` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:42.720623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f266029fd86485f970d82b4a68de060)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:42.735400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:42.781754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 891/1764] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:42 2022] POST /identity/v3/projects => generated 385 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:42.786422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5171a3-2659-4482-94d8-fccad2039836 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:42.790593 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-df5171a3-2659-4482-94d8-fccad2039836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:42.844262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f266029fd86485f970d82b4a68de060', 'request_id': 'req-14fe0cd0-ea16-4904-82fc-94fe96dd823d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:42.854589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f266029fd86485f970d82b4a68de060', 'request_id': 'req-14fe0cd0-ea16-4904-82fc-94fe96dd823d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:42.881778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f266029fd86485f970d82b4a68de060', 'request_id': 'req-14fe0cd0-ea16-4904-82fc-94fe96dd823d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:42.903971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] The token cache is being invalidate because project 2f266029fd86485f970d82b4a68de060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:13:42.904204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:42.904922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:42.910409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 874/1765] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:13:42 2022] DELETE /identity/v3/projects/2f266029fd86485f970d82b4a68de060 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:13:42.911611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:42.913709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:42.918289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5171a3-2659-4482-94d8-fccad2039836 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:42.952659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:42.960909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:43.019856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccd310-f2d5-46f5-8444-4a97a112f723 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:43.020762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccd310-f2d5-46f5-8444-4a97a112f723 service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:43.021196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccd310-f2d5-46f5-8444-4a97a112f723 service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:43.021618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccd310-f2d5-46f5-8444-4a97a112f723 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:43.023897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-93ccd310-f2d5-46f5-8444-4a97a112f723 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:43.046537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe0cd0-ea16-4904-82fc-94fe96dd823d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:43.052408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5bd89-6a5b-460d-af7d-c29d1afa71ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:43.053106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5bd89-6a5b-460d-af7d-c29d1afa71ea admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:43.053554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5bd89-6a5b-460d-af7d-c29d1afa71ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:43.053840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5bd89-6a5b-460d-af7d-c29d1afa71ea admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:43.054546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5bd89-6a5b-460d-af7d-c29d1afa71ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:43.056652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5bd89-6a5b-460d-af7d-c29d1afa71ea admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:43.092019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 875/1766] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:42 2022] POST /identity/v3/users => generated 299 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:43.094027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5bd89-6a5b-460d-af7d-c29d1afa71ea admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:43.099752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fea5bd89-6a5b-460d-af7d-c29d1afa71ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:43.132879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5bd89-6a5b-460d-af7d-c29d1afa71ea admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:43.135294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccd310-f2d5-46f5-8444-4a97a112f723 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:43.138314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccd310-f2d5-46f5-8444-4a97a112f723 service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:43.149425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:43.150813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:43.151313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:43.151754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] PATH_INFO: `/v3/users/2660d757582146f59eeaf493c55f8e79` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:43.153644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2660d757582146f59eeaf493c55f8e79)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:43.156458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:43.162470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 892/1767] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:13:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:13:43.163355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccd310-f2d5-46f5-8444-4a97a112f723 service swift] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:43.181788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccd310-f2d5-46f5-8444-4a97a112f723 service swift] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:43.252385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3eb5-0965-4e60-8ea6-e1f22e955195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:43.256696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3eb5-0965-4e60-8ea6-e1f22e955195 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:43.257952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3eb5-0965-4e60-8ea6-e1f22e955195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:43.258245 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3eb5-0965-4e60-8ea6-e1f22e955195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:43.271366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d3eb5-0965-4e60-8ea6-e1f22e955195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:43.271663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d3eb5-0965-4e60-8ea6-e1f22e955195 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:43.300503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2660d757582146f59eeaf493c55f8e79'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:43.304539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2660d757582146f59eeaf493c55f8e79'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:43.305747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2660d757582146f59eeaf493c55f8e79'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:43.313908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 893/1768] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:43.319160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3eb5-0965-4e60-8ea6-e1f22e955195 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:43.321948 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-eb6d3eb5-0965-4e60-8ea6-e1f22e955195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:43.333549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2660d757582146f59eeaf493c55f8e79'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:43.382375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2660d757582146f59eeaf493c55f8e79'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:43.385163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3eb5-0965-4e60-8ea6-e1f22e955195 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:43.402173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35db410-ec1d-4c78-b8dc-6714e54e2f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:43.405999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35db410-ec1d-4c78-b8dc-6714e54e2f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:43.405999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35db410-ec1d-4c78-b8dc-6714e54e2f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:43.405999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35db410-ec1d-4c78-b8dc-6714e54e2f0a admin admin] PATH_INFO: `/v3/projects/312ba9e3509845a8a9cc2e0b5f532760/users/4249f597089b417aa42c40263fec9ebc/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:43.448480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 876/1769] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:13:43 2022] DELETE /identity/v3/users/2660d757582146f59eeaf493c55f8e79 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:43.458473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35db410-ec1d-4c78-b8dc-6714e54e2f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=312ba9e3509845a8a9cc2e0b5f532760, user_id=4249f597089b417aa42c40263fec9ebc, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:43.469887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35db410-ec1d-4c78-b8dc-6714e54e2f0a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:43.552560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 894/1770] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:43 2022] PUT /identity/v3/projects/312ba9e3509845a8a9cc2e0b5f532760/users/4249f597089b417aa42c40263fec9ebc/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:13:43.556670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:43.870209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbbfc8-04d7-41ea-9609-7dedef83e6f7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:44.001517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:44.062114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c3969-1632-465a-a7ed-91f4961bdaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:44.063174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c3969-1632-465a-a7ed-91f4961bdaeb admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:44.064411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c3969-1632-465a-a7ed-91f4961bdaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:44.065840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c3969-1632-465a-a7ed-91f4961bdaeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:44.067874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c3969-1632-465a-a7ed-91f4961bdaeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:44.072675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c3969-1632-465a-a7ed-91f4961bdaeb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:44.108581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35db410-ec1d-4c78-b8dc-6714e54e2f0a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:44.109199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 877/1771] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 552 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:44.110267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c3969-1632-465a-a7ed-91f4961bdaeb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:44.111568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b35db410-ec1d-4c78-b8dc-6714e54e2f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:44.125003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c3969-1632-465a-a7ed-91f4961bdaeb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:44.140704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5839a-60b6-42e0-8d1e-4fb58894d06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:44.141211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35db410-ec1d-4c78-b8dc-6714e54e2f0a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:44.141623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5839a-60b6-42e0-8d1e-4fb58894d06c admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:44.141909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5839a-60b6-42e0-8d1e-4fb58894d06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:44.142195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5839a-60b6-42e0-8d1e-4fb58894d06c admin admin] PATH_INFO: `/v3/projects/312ba9e3509845a8a9cc2e0b5f532760/users/4249f597089b417aa42c40263fec9ebc/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:44.145715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5839a-60b6-42e0-8d1e-4fb58894d06c admin admin] RBAC: Authorizing `identity:create_grant(project_id=312ba9e3509845a8a9cc2e0b5f532760, user_id=4249f597089b417aa42c40263fec9ebc, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:44.149321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd5839a-60b6-42e0-8d1e-4fb58894d06c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:44.162204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:44.166077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:44.166430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:44.166728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] PATH_INFO: `/v3/projects/cfdc3334eeb94e6b8c59dbe56d15f0bf` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:44.175099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfdc3334eeb94e6b8c59dbe56d15f0bf)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:44.181130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:44.183970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 878/1772] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:44 2022] PUT /identity/v3/projects/312ba9e3509845a8a9cc2e0b5f532760/users/4249f597089b417aa42c40263fec9ebc/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:13:44.196834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a59d-3b12-4d89-a15c-0ffe9bf58507 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:44.197241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a59d-3b12-4d89-a15c-0ffe9bf58507 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:44.197883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601a59d-3b12-4d89-a15c-0ffe9bf58507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:44.294777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3601a59d-3b12-4d89-a15c-0ffe9bf58507 None None] MFA Rules not processed for user `4249f597089b417aa42c40263fec9ebc`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:44.298163 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3601a59d-3b12-4d89-a15c-0ffe9bf58507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:44.300799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfdc3334eeb94e6b8c59dbe56d15f0bf', 'request_id': 'req-6f7ed296-65e0-4a5f-846c-a118e0545aee'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:44.318851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfdc3334eeb94e6b8c59dbe56d15f0bf', 'request_id': 'req-6f7ed296-65e0-4a5f-846c-a118e0545aee'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:44.342368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfdc3334eeb94e6b8c59dbe56d15f0bf', 'request_id': 'req-6f7ed296-65e0-4a5f-846c-a118e0545aee'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:44.376670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] The token cache is being invalidate because project cfdc3334eeb94e6b8c59dbe56d15f0bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:13:44.378717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:44.382511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6f7ed296-65e0-4a5f-846c-a118e0545aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:44.389357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 895/1773] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:13:44 2022] DELETE /identity/v3/projects/cfdc3334eeb94e6b8c59dbe56d15f0bf => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:44.395431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858480b3-e6e5-4c90-aa89-27f393b5a01c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:44.396093 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858480b3-e6e5-4c90-aa89-27f393b5a01c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:44.397399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858480b3-e6e5-4c90-aa89-27f393b5a01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:44.449230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-858480b3-e6e5-4c90-aa89-27f393b5a01c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:44.452578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-858480b3-e6e5-4c90-aa89-27f393b5a01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:44.490211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:44.515980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 879/1774] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:44.522203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601a59d-3b12-4d89-a15c-0ffe9bf58507 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:44.524546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3601a59d-3b12-4d89-a15c-0ffe9bf58507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:44.637612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 896/1775] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:44.646937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb2750-f2ca-46c1-ac75-ad8fe16387ab None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:44.647305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb2750-f2ca-46c1-ac75-ad8fe16387ab None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:44.647641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb2750-f2ca-46c1-ac75-ad8fe16387ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:44.673645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601a59d-3b12-4d89-a15c-0ffe9bf58507 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:44.688693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa497faf-8a42-4cb3-a09c-7e08a93f05ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:44.690326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497faf-8a42-4cb3-a09c-7e08a93f05ba service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:44.692736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497faf-8a42-4cb3-a09c-7e08a93f05ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:44.693089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa497faf-8a42-4cb3-a09c-7e08a93f05ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:44.696225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-6bfb2750-f2ca-46c1-ac75-ad8fe16387ab None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:44.697526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6bfb2750-f2ca-46c1-ac75-ad8fe16387ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:44.717537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 897/1776] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:13:44.721603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb2750-f2ca-46c1-ac75-ad8fe16387ab None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:44.725861 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa497faf-8a42-4cb3-a09c-7e08a93f05ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:44.728180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa497faf-8a42-4cb3-a09c-7e08a93f05ba service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:44.752080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb2750-f2ca-46c1-ac75-ad8fe16387ab None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:44.753796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 880/1777] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:44.759128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17847a5b-54b2-4ae4-82e6-faaa6afe03e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:44.760016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17847a5b-54b2-4ae4-82e6-faaa6afe03e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:44.760480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17847a5b-54b2-4ae4-82e6-faaa6afe03e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:44.760856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17847a5b-54b2-4ae4-82e6-faaa6afe03e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:44.761964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17847a5b-54b2-4ae4-82e6-faaa6afe03e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:44.762464 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17847a5b-54b2-4ae4-82e6-faaa6afe03e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:44.764642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17847a5b-54b2-4ae4-82e6-faaa6afe03e0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:44.788962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 898/1778] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:44.791726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa497faf-8a42-4cb3-a09c-7e08a93f05ba service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:44.815192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa497faf-8a42-4cb3-a09c-7e08a93f05ba service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:44.825141 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56c156-9691-4494-b590-c6dbb1b98853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:44.828899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c156-9691-4494-b590-c6dbb1b98853 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:44.830293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c156-9691-4494-b590-c6dbb1b98853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:44.830663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c156-9691-4494-b590-c6dbb1b98853 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:44.833942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56c156-9691-4494-b590-c6dbb1b98853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:44.842516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56c156-9691-4494-b590-c6dbb1b98853 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:44.904817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17847a5b-54b2-4ae4-82e6-faaa6afe03e0 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:44.905267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 881/1779] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:44 2022] POST /identity/v3/projects => generated 401 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:13:44.922128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17847a5b-54b2-4ae4-82e6-faaa6afe03e0 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:45.107388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cab6b7-f7d6-4dff-90ca-0e1f59273a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:45.110137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cab6b7-f7d6-4dff-90ca-0e1f59273a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:45.110821 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cab6b7-f7d6-4dff-90ca-0e1f59273a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:45.112746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cab6b7-f7d6-4dff-90ca-0e1f59273a23 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:45.113792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cab6b7-f7d6-4dff-90ca-0e1f59273a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:45.116473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cab6b7-f7d6-4dff-90ca-0e1f59273a23 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:45.172591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56c156-9691-4494-b590-c6dbb1b98853 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:45.188258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 899/1780] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:44 2022] POST /identity/v3/users => generated 301 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:45.188910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56c156-9691-4494-b590-c6dbb1b98853 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:45.199289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe52a8-3944-4589-9e13-ff3c68fb82a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:45.204545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe52a8-3944-4589-9e13-ff3c68fb82a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:45.204545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe52a8-3944-4589-9e13-ff3c68fb82a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:45.204545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe52a8-3944-4589-9e13-ff3c68fb82a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:45.204545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe52a8-3944-4589-9e13-ff3c68fb82a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:45.204545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe52a8-3944-4589-9e13-ff3c68fb82a1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:45.239088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cab6b7-f7d6-4dff-90ca-0e1f59273a23 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:45.256490 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 882/1781] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:45.258630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cab6b7-f7d6-4dff-90ca-0e1f59273a23 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:45.272290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c8ffa-63dd-41f6-ae30-135a4bd2d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:45.273233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c8ffa-63dd-41f6-ae30-135a4bd2d89c admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:45.273686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c8ffa-63dd-41f6-ae30-135a4bd2d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:45.274124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c8ffa-63dd-41f6-ae30-135a4bd2d89c admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:45.275105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c8ffa-63dd-41f6-ae30-135a4bd2d89c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:45.277376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c8ffa-63dd-41f6-ae30-135a4bd2d89c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:45.294493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 900/1782] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:45.298739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe52a8-3944-4589-9e13-ff3c68fb82a1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:45.313342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe52a8-3944-4589-9e13-ff3c68fb82a1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:45.322147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff43c7-43ec-4171-8e94-bcbc428a769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:45.323257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff43c7-43ec-4171-8e94-bcbc428a769b admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:45.323643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff43c7-43ec-4171-8e94-bcbc428a769b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:45.324018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff43c7-43ec-4171-8e94-bcbc428a769b admin admin] PATH_INFO: `/v3/projects/004171b6113d444396ba9b0985462401/users/f5b34b5b588f4b67b3addfe877854ae6/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:45.334344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:45.353385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff43c7-43ec-4171-8e94-bcbc428a769b admin admin] RBAC: Authorizing `identity:create_grant(project_id=004171b6113d444396ba9b0985462401, user_id=f5b34b5b588f4b67b3addfe877854ae6, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:45.356233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff43c7-43ec-4171-8e94-bcbc428a769b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:45.378670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 883/1783] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:45 2022] PUT /identity/v3/projects/004171b6113d444396ba9b0985462401/users/f5b34b5b588f4b67b3addfe877854ae6/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:13:45.389962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cada7-22d4-4821-8fec-ced0bbc5b8d0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:45.390247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cada7-22d4-4821-8fec-ced0bbc5b8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:45.390526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cada7-22d4-4821-8fec-ced0bbc5b8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:45.484751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0f3cada7-22d4-4821-8fec-ced0bbc5b8d0 None None] MFA Rules not processed for user `f5b34b5b588f4b67b3addfe877854ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:45.488905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0f3cada7-22d4-4821-8fec-ced0bbc5b8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:45.688073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 901/1784] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:45.692381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff43c7-43ec-4171-8e94-bcbc428a769b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:45.834446 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff43c7-43ec-4171-8e94-bcbc428a769b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:45.839893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3db79d-1b98-4142-afba-b97b7446153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:45.840641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db79d-1b98-4142-afba-b97b7446153c admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:45.840930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db79d-1b98-4142-afba-b97b7446153c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:45.841445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3db79d-1b98-4142-afba-b97b7446153c admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:45.842244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3db79d-1b98-4142-afba-b97b7446153c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:45.844545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3db79d-1b98-4142-afba-b97b7446153c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:45.873075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cada7-22d4-4821-8fec-ced0bbc5b8d0 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:45.875036 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0f3cada7-22d4-4821-8fec-ced0bbc5b8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:45.898723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cada7-22d4-4821-8fec-ced0bbc5b8d0 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:45.902223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 884/1785] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:45 2022] POST /identity/v3/projects => generated 401 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:45.905956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3db79d-1b98-4142-afba-b97b7446153c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:45.913683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:46.049685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.051669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.051669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.055333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] PATH_INFO: `/v3/projects/2a1dd854625b4334bf5130fea9877a3d` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.059355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a1dd854625b4334bf5130fea9877a3d)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.061446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:46.078876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3db79d-1b98-4142-afba-b97b7446153c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.086048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd29d46-22e1-40d9-b9d0-506762710de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.087249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd29d46-22e1-40d9-b9d0-506762710de1 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.087841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd29d46-22e1-40d9-b9d0-506762710de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.099740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd29d46-22e1-40d9-b9d0-506762710de1 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.100468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd29d46-22e1-40d9-b9d0-506762710de1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.102385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd29d46-22e1-40d9-b9d0-506762710de1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:46.140365 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 885/1786] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:45 2022] POST /identity/v3/users => generated 307 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:46.141102 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd29d46-22e1-40d9-b9d0-506762710de1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:46.144499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-afd29d46-22e1-40d9-b9d0-506762710de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:46.168940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a1dd854625b4334bf5130fea9877a3d', 'request_id': 'req-3516290b-01b2-4ef0-bf1a-2e73889aaf76'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:46.179391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a1dd854625b4334bf5130fea9877a3d', 'request_id': 'req-3516290b-01b2-4ef0-bf1a-2e73889aaf76'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:46.186460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a1dd854625b4334bf5130fea9877a3d', 'request_id': 'req-3516290b-01b2-4ef0-bf1a-2e73889aaf76'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:46.209691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] The token cache is being invalidate because project 2a1dd854625b4334bf5130fea9877a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:13:46.209987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:46.210624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:13:46.213319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 902/1787] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:13:45 2022] DELETE /identity/v3/projects/2a1dd854625b4334bf5130fea9877a3d => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:46.215878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:46.218075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:46.220867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd29d46-22e1-40d9-b9d0-506762710de1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.232296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:46.284811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b549d-e739-4189-bc0f-eabd7325bbfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.285075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b549d-e739-4189-bc0f-eabd7325bbfb service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.285937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b549d-e739-4189-bc0f-eabd7325bbfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.286225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b549d-e739-4189-bc0f-eabd7325bbfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.288373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c58b549d-e739-4189-bc0f-eabd7325bbfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:46.306466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3516290b-01b2-4ef0-bf1a-2e73889aaf76 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.312017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeae563-874c-4a27-80c9-7f1a2b2a544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.312519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeae563-874c-4a27-80c9-7f1a2b2a544c admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.312876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeae563-874c-4a27-80c9-7f1a2b2a544c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.313327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeae563-874c-4a27-80c9-7f1a2b2a544c admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.314091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeae563-874c-4a27-80c9-7f1a2b2a544c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.315909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeae563-874c-4a27-80c9-7f1a2b2a544c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:46.334099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 903/1788] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:46.335910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fce3df-64d7-4d92-b05b-831dd2cf9859 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.336604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fce3df-64d7-4d92-b05b-831dd2cf9859 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.337022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fce3df-64d7-4d92-b05b-831dd2cf9859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.373017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-f9fce3df-64d7-4d92-b05b-831dd2cf9859 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:46.374279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f9fce3df-64d7-4d92-b05b-831dd2cf9859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:46.383272 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 904/1789] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:46.383895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fce3df-64d7-4d92-b05b-831dd2cf9859 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:46.387238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b549d-e739-4189-bc0f-eabd7325bbfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.389724 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b549d-e739-4189-bc0f-eabd7325bbfb service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:46.396165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fce3df-64d7-4d92-b05b-831dd2cf9859 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.403766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ec13e-48d3-46cd-b022-3ac125aebea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.404682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ec13e-48d3-46cd-b022-3ac125aebea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.405066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ec13e-48d3-46cd-b022-3ac125aebea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.405232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ec13e-48d3-46cd-b022-3ac125aebea2 admin admin] PATH_INFO: `/v3/projects/8fd0d64ab4d8422f8df93274895d0ec9/users/9feef3a703b14b1881a6aeff302e5e1d/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.418925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 886/1790] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:46.425718 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ec13e-48d3-46cd-b022-3ac125aebea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd0d64ab4d8422f8df93274895d0ec9, user_id=9feef3a703b14b1881a6aeff302e5e1d, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.431767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ec13e-48d3-46cd-b022-3ac125aebea2 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:46.442785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d005c96-beaa-4d3a-94b3-025e9611fb9c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.443005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d005c96-beaa-4d3a-94b3-025e9611fb9c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.443298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d005c96-beaa-4d3a-94b3-025e9611fb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.462820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 905/1791] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:46 2022] PUT /identity/v3/projects/8fd0d64ab4d8422f8df93274895d0ec9/users/9feef3a703b14b1881a6aeff302e5e1d/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:46.464962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ec13e-48d3-46cd-b022-3ac125aebea2 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:46.468166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2d005c96-beaa-4d3a-94b3-025e9611fb9c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:46.469655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2d005c96-beaa-4d3a-94b3-025e9611fb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:46.479439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:46.549828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ec13e-48d3-46cd-b022-3ac125aebea2 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.557235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05b219-6d5a-4c13-9b60-9ec826fbe9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.557985 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05b219-6d5a-4c13-9b60-9ec826fbe9fa admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.558280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05b219-6d5a-4c13-9b60-9ec826fbe9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.558565 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b05b219-6d5a-4c13-9b60-9ec826fbe9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.559650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05b219-6d5a-4c13-9b60-9ec826fbe9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.563105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b05b219-6d5a-4c13-9b60-9ec826fbe9fa admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:46.575951 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 906/1792] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:46.579917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05b219-6d5a-4c13-9b60-9ec826fbe9fa admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:46.586240 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 887/1793] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:46.593671 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d005c96-beaa-4d3a-94b3-025e9611fb9c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:46.595817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b05b219-6d5a-4c13-9b60-9ec826fbe9fa admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.601129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1beb57-48c3-4660-9c88-50472c192eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.601995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1beb57-48c3-4660-9c88-50472c192eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.602297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1beb57-48c3-4660-9c88-50472c192eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.602562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1beb57-48c3-4660-9c88-50472c192eb5 admin admin] PATH_INFO: `/v3/projects/8fd0d64ab4d8422f8df93274895d0ec9/users/9feef3a703b14b1881a6aeff302e5e1d/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.604759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1beb57-48c3-4660-9c88-50472c192eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd0d64ab4d8422f8df93274895d0ec9, user_id=9feef3a703b14b1881a6aeff302e5e1d, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.607547 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1beb57-48c3-4660-9c88-50472c192eb5 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:46.620985 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 907/1794] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:46 2022] PUT /identity/v3/projects/8fd0d64ab4d8422f8df93274895d0ec9/users/9feef3a703b14b1881a6aeff302e5e1d/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:46.623336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d005c96-beaa-4d3a-94b3-025e9611fb9c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.632428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bec40f-0594-4d44-a51d-a28b8259ee6a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.632730 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bec40f-0594-4d44-a51d-a28b8259ee6a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.633002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bec40f-0594-4d44-a51d-a28b8259ee6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.680282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-35bec40f-0594-4d44-a51d-a28b8259ee6a None None] MFA Rules not processed for user `9feef3a703b14b1881a6aeff302e5e1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:46.682381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-35bec40f-0594-4d44-a51d-a28b8259ee6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:46.734536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b20045-59c9-4571-a27b-faaa583f8116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.736154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b20045-59c9-4571-a27b-faaa583f8116 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.736440 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b20045-59c9-4571-a27b-faaa583f8116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.736813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b20045-59c9-4571-a27b-faaa583f8116 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.740752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b20045-59c9-4571-a27b-faaa583f8116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:13:46.741975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b20045-59c9-4571-a27b-faaa583f8116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.747898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b20045-59c9-4571-a27b-faaa583f8116 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:46.766988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 888/1795] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:13:46.770073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b20045-59c9-4571-a27b-faaa583f8116 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:46.778823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:46.781985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b20045-59c9-4571-a27b-faaa583f8116 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.792003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02653a5-c4b8-4f2a-8c47-db6674c2b518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.792682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02653a5-c4b8-4f2a-8c47-db6674c2b518 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.792970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02653a5-c4b8-4f2a-8c47-db6674c2b518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.793251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02653a5-c4b8-4f2a-8c47-db6674c2b518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.793960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02653a5-c4b8-4f2a-8c47-db6674c2b518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.795923 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02653a5-c4b8-4f2a-8c47-db6674c2b518 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:46.826622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 908/1796] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:46 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:46.840508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bec40f-0594-4d44-a51d-a28b8259ee6a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:46.841574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-35bec40f-0594-4d44-a51d-a28b8259ee6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:46.844101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 889/1797] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:46 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:46.848452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02653a5-c4b8-4f2a-8c47-db6674c2b518 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:46.933680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bec40f-0594-4d44-a51d-a28b8259ee6a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.938041 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02653a5-c4b8-4f2a-8c47-db6674c2b518 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.944460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd9689-2bb1-47a6-8d94-63cb039ea95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.945150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd9689-2bb1-47a6-8d94-63cb039ea95f admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.945437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd9689-2bb1-47a6-8d94-63cb039ea95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.945719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd9689-2bb1-47a6-8d94-63cb039ea95f admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.946424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd9689-2bb1-47a6-8d94-63cb039ea95f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.948432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd9689-2bb1-47a6-8d94-63cb039ea95f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:46.971090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 890/1798] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:46 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:46.973455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd9689-2bb1-47a6-8d94-63cb039ea95f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:46.975487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:46.984197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd9689-2bb1-47a6-8d94-63cb039ea95f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:46.991671 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde4b04-437b-49fb-a56a-3e2787c3aea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:46.992778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde4b04-437b-49fb-a56a-3e2787c3aea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:46.993128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde4b04-437b-49fb-a56a-3e2787c3aea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:46.993443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edde4b04-437b-49fb-a56a-3e2787c3aea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:46.994403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde4b04-437b-49fb-a56a-3e2787c3aea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:46.996656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edde4b04-437b-49fb-a56a-3e2787c3aea6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.001876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f1b5c-6956-407d-b099-27f3052ebb3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:47.002575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f1b5c-6956-407d-b099-27f3052ebb3a service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.002870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f1b5c-6956-407d-b099-27f3052ebb3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.003182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f1b5c-6956-407d-b099-27f3052ebb3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.006977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 891/1799] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:47.010895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde4b04-437b-49fb-a56a-3e2787c3aea6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:47.022000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edde4b04-437b-49fb-a56a-3e2787c3aea6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:47.027505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef23f1-d679-4860-b67c-922bc597b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:47.028290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef23f1-d679-4860-b67c-922bc597b7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.028940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef23f1-d679-4860-b67c-922bc597b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.029319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef23f1-d679-4860-b67c-922bc597b7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.030150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef23f1-d679-4860-b67c-922bc597b7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:47.034185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef23f1-d679-4860-b67c-922bc597b7ff admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.050326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 892/1800] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:13:47.052364 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef23f1-d679-4860-b67c-922bc597b7ff admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:47.063580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef23f1-d679-4860-b67c-922bc597b7ff admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:47.070000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca96a51-281a-46c9-8a5f-8fe34d55fa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:47.070000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca96a51-281a-46c9-8a5f-8fe34d55fa5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.070539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca96a51-281a-46c9-8a5f-8fe34d55fa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.070954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca96a51-281a-46c9-8a5f-8fe34d55fa5a admin admin] PATH_INFO: `/v3/projects/bca19d45972e4eb692a8a74156278e3b/users/8e7daef95586424b8f812ad5ddcf3657/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.094418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca96a51-281a-46c9-8a5f-8fe34d55fa5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca19d45972e4eb692a8a74156278e3b, user_id=8e7daef95586424b8f812ad5ddcf3657, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:47.097892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca96a51-281a-46c9-8a5f-8fe34d55fa5a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.116349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 893/1801] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:47 2022] PUT /identity/v3/projects/bca19d45972e4eb692a8a74156278e3b/users/8e7daef95586424b8f812ad5ddcf3657/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:47.128695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51b2be-0b93-44dc-bc12-72390fc303e6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.129017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51b2be-0b93-44dc-bc12-72390fc303e6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.129825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51b2be-0b93-44dc-bc12-72390fc303e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.162604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fc51b2be-0b93-44dc-bc12-72390fc303e6 None None] MFA Rules not processed for user `8e7daef95586424b8f812ad5ddcf3657`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:47.163845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fc51b2be-0b93-44dc-bc12-72390fc303e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:47.212707 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f1b5c-6956-407d-b099-27f3052ebb3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:47.214971 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f1b5c-6956-407d-b099-27f3052ebb3a service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.226890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 909/1802] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:46 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:47.288473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 894/1803] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:13:47.292880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f1b5c-6956-407d-b099-27f3052ebb3a service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:47.304470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f1b5c-6956-407d-b099-27f3052ebb3a service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:47.333575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51b2be-0b93-44dc-bc12-72390fc303e6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:47.352665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:47.375982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a136b-f181-4b27-853c-5307bd0a088b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:47.377095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a136b-f181-4b27-853c-5307bd0a088b admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.377386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a136b-f181-4b27-853c-5307bd0a088b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.377665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a136b-f181-4b27-853c-5307bd0a088b admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.378832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a136b-f181-4b27-853c-5307bd0a088b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:47.381405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a136b-f181-4b27-853c-5307bd0a088b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.392514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51b2be-0b93-44dc-bc12-72390fc303e6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:47.398113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c80e-ade3-4346-a33c-87b4fe5b1863 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:47.399087 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c80e-ade3-4346-a33c-87b4fe5b1863 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.399463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c80e-ade3-4346-a33c-87b4fe5b1863 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.399875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c80e-ade3-4346-a33c-87b4fe5b1863 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.402036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9a16c80e-ade3-4346-a33c-87b4fe5b1863 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:47.406699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 910/1804] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:13:47 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:47.410745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a136b-f181-4b27-853c-5307bd0a088b admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:47.463898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16c80e-ade3-4346-a33c-87b4fe5b1863 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:47.465277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16c80e-ade3-4346-a33c-87b4fe5b1863 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.472289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a136b-f181-4b27-853c-5307bd0a088b admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:47.475026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 895/1805] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:47 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:13:47.477868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b7e69-a59e-4d7e-8035-5d0a731210a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:47.478543 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7e69-a59e-4d7e-8035-5d0a731210a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.478825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7e69-a59e-4d7e-8035-5d0a731210a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.479158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b7e69-a59e-4d7e-8035-5d0a731210a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.479850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b7e69-a59e-4d7e-8035-5d0a731210a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:47.481758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b7e69-a59e-4d7e-8035-5d0a731210a9 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.492453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:47.503384 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 911/1806] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:13:47 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:13:47.508015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c80e-ade3-4346-a33c-87b4fe5b1863 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:47.519138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c80e-ade3-4346-a33c-87b4fe5b1863 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:47.523872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a0b44-4bbc-45c7-98d1-6919cc4a3763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:47.524539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a0b44-4bbc-45c7-98d1-6919cc4a3763 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.524826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a0b44-4bbc-45c7-98d1-6919cc4a3763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.525117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a0b44-4bbc-45c7-98d1-6919cc4a3763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.525831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a0b44-4bbc-45c7-98d1-6919cc4a3763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:47.527459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a0b44-4bbc-45c7-98d1-6919cc4a3763 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.537851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 896/1807] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:13:47.541968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b7e69-a59e-4d7e-8035-5d0a731210a9 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:47.552072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b7e69-a59e-4d7e-8035-5d0a731210a9 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:47.557679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987027a6-a83f-455f-8fbb-099dd91b7937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:47.558484 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987027a6-a83f-455f-8fbb-099dd91b7937 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.558772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987027a6-a83f-455f-8fbb-099dd91b7937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.559076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987027a6-a83f-455f-8fbb-099dd91b7937 admin admin] PATH_INFO: `/v3/projects/ddfd5b17b3ef4e99b880599b7403ee4a/users/ba19da460f7d45cc985d23171efc51f5/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.575984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987027a6-a83f-455f-8fbb-099dd91b7937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddfd5b17b3ef4e99b880599b7403ee4a, user_id=ba19da460f7d45cc985d23171efc51f5, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:47.578760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987027a6-a83f-455f-8fbb-099dd91b7937 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.592552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 912/1808] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:47 2022] PUT /identity/v3/projects/ddfd5b17b3ef4e99b880599b7403ee4a/users/ba19da460f7d45cc985d23171efc51f5/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:47.596546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a0b44-4bbc-45c7-98d1-6919cc4a3763 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:47.607673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a0b44-4bbc-45c7-98d1-6919cc4a3763 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:47.680107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0a9d0-2f1f-4051-997f-4ad410f7f622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:47.684968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0a9d0-2f1f-4051-997f-4ad410f7f622 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.684968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0a9d0-2f1f-4051-997f-4ad410f7f622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.684968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0a9d0-2f1f-4051-997f-4ad410f7f622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.684968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0a9d0-2f1f-4051-997f-4ad410f7f622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:47.687722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0a9d0-2f1f-4051-997f-4ad410f7f622 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.708978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 897/1809] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:13:47.710467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987027a6-a83f-455f-8fbb-099dd91b7937 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:47.727101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987027a6-a83f-455f-8fbb-099dd91b7937 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:47.735374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade158a7-b0b9-4d49-a86b-267fa3b29df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:47.735709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade158a7-b0b9-4d49-a86b-267fa3b29df4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.736137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade158a7-b0b9-4d49-a86b-267fa3b29df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.736575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade158a7-b0b9-4d49-a86b-267fa3b29df4 admin admin] PATH_INFO: `/v3/projects/ddfd5b17b3ef4e99b880599b7403ee4a/users/ba19da460f7d45cc985d23171efc51f5/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.739947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade158a7-b0b9-4d49-a86b-267fa3b29df4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddfd5b17b3ef4e99b880599b7403ee4a, user_id=ba19da460f7d45cc985d23171efc51f5, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:47.747201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade158a7-b0b9-4d49-a86b-267fa3b29df4 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:47.784600 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 913/1810] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:13:47 2022] PUT /identity/v3/projects/ddfd5b17b3ef4e99b880599b7403ee4a/users/ba19da460f7d45cc985d23171efc51f5/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:13:47.788113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93339-5ee9-45e9-aa0d-37ddc2cd1519 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:47.788521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93339-5ee9-45e9-aa0d-37ddc2cd1519 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:47.788931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c93339-5ee9-45e9-aa0d-37ddc2cd1519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:47.830024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-63c93339-5ee9-45e9-aa0d-37ddc2cd1519 None None] MFA Rules not processed for user `ba19da460f7d45cc985d23171efc51f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:47.833944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-63c93339-5ee9-45e9-aa0d-37ddc2cd1519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:47.934517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:47.974615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 898/1811] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:13:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:47.990053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade158a7-b0b9-4d49-a86b-267fa3b29df4 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:47.992318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ade158a7-b0b9-4d49-a86b-267fa3b29df4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:48.085215 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade158a7-b0b9-4d49-a86b-267fa3b29df4 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:48.089992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d617379-f145-4bb5-85fe-ef7e029a88cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:48.090774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617379-f145-4bb5-85fe-ef7e029a88cf service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:48.091300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617379-f145-4bb5-85fe-ef7e029a88cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:48.091582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d617379-f145-4bb5-85fe-ef7e029a88cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:48.102479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d617379-f145-4bb5-85fe-ef7e029a88cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:48.103967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d617379-f145-4bb5-85fe-ef7e029a88cf service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:48.115338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 914/1812] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:48.194338 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c93339-5ee9-45e9-aa0d-37ddc2cd1519 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:48.204497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c93339-5ee9-45e9-aa0d-37ddc2cd1519 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:48.208181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdef44-072d-40ab-8fa7-10e2cb6e14ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:48.208835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdef44-072d-40ab-8fa7-10e2cb6e14ca service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:48.209128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdef44-072d-40ab-8fa7-10e2cb6e14ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:48.209406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bdef44-072d-40ab-8fa7-10e2cb6e14ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:48.287667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bdef44-072d-40ab-8fa7-10e2cb6e14ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:48.289427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bdef44-072d-40ab-8fa7-10e2cb6e14ca service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:48.300498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 899/1813] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:13:58.110241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=43,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:58.295172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=37 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:13:58.942888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c365a-4f33-478c-97e3-3d0341b20352 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:58.943394 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c365a-4f33-478c-97e3-3d0341b20352 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:58.943840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c365a-4f33-478c-97e3-3d0341b20352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:58.963665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-a90c365a-4f33-478c-97e3-3d0341b20352 None None] MFA Rules not processed for user `cd7fa93466c54632ad95834297410e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:13:58.964874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a90c365a-4f33-478c-97e3-3d0341b20352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:59.019962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 915/1814] 10.222.0.33 () {58 vars in 1015 bytes} [Tue Jul 5 15:13:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:13:59.044094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdef44-072d-40ab-8fa7-10e2cb6e14ca service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:59.046246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-61bdef44-072d-40ab-8fa7-10e2cb6e14ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:59.047169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c365a-4f33-478c-97e3-3d0341b20352 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:59.048653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a90c365a-4f33-478c-97e3-3d0341b20352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:59.063507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bdef44-072d-40ab-8fa7-10e2cb6e14ca service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:59.065230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c365a-4f33-478c-97e3-3d0341b20352 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:59.068631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00428f-9a56-40c6-a522-c44e02f4af9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:59.069386 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00428f-9a56-40c6-a522-c44e02f4af9c service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:59.069621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ed794-dcc2-4fc0-a57d-364b6f67abb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:59.069909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00428f-9a56-40c6-a522-c44e02f4af9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:59.070280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ed794-dcc2-4fc0-a57d-364b6f67abb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:59.070404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00428f-9a56-40c6-a522-c44e02f4af9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:59.070564 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ed794-dcc2-4fc0-a57d-364b6f67abb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:59.070848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ed794-dcc2-4fc0-a57d-364b6f67abb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:59.080866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ed794-dcc2-4fc0-a57d-364b6f67abb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:59.081406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00428f-9a56-40c6-a522-c44e02f4af9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:59.082572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ed794-dcc2-4fc0-a57d-364b6f67abb0 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:59.083313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00428f-9a56-40c6-a522-c44e02f4af9c service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:59.095291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 900/1815] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:13:59.096281 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00428f-9a56-40c6-a522-c44e02f4af9c service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:13:59.097899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1a00428f-9a56-40c6-a522-c44e02f4af9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:13:59.106259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 916/1816] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:13:59.114630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00428f-9a56-40c6-a522-c44e02f4af9c service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:13:59.121312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88340731-84bb-46ba-8da5-3973df91844f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:13:59.122299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88340731-84bb-46ba-8da5-3973df91844f service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:13:59.123366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88340731-84bb-46ba-8da5-3973df91844f service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:13:59.123831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88340731-84bb-46ba-8da5-3973df91844f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:13:59.137356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88340731-84bb-46ba-8da5-3973df91844f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:13:59.139804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88340731-84bb-46ba-8da5-3973df91844f service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:13:59.152307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 901/1817] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:13:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:00.604665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c148bf5-2e6d-45ef-9d9a-bafee2af0bd4 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:00.605168 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c148bf5-2e6d-45ef-9d9a-bafee2af0bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:00.605264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c148bf5-2e6d-45ef-9d9a-bafee2af0bd4 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:00.606629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 917/1818] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:14:00.713733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081183d5-f5e2-47f2-a13e-32a22e1ca014 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:00.714293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081183d5-f5e2-47f2-a13e-32a22e1ca014 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:00.714471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081183d5-f5e2-47f2-a13e-32a22e1ca014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:00.762449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-081183d5-f5e2-47f2-a13e-32a22e1ca014 None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:00.763964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-081183d5-f5e2-47f2-a13e-32a22e1ca014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:00.775651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c148bf5-2e6d-45ef-9d9a-bafee2af0bd4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:00.796564 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c148bf5-2e6d-45ef-9d9a-bafee2af0bd4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:00.809604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbcac8-1626-4594-b107-a770723ee27a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:00.811163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbcac8-1626-4594-b107-a770723ee27a service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:00.815870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbcac8-1626-4594-b107-a770723ee27a service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:00.816467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbcac8-1626-4594-b107-a770723ee27a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:00.819376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a0bbcac8-1626-4594-b107-a770723ee27a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:00.962186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 902/1819] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:14:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:14:00.977309 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbcac8-1626-4594-b107-a770723ee27a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:00.979122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbcac8-1626-4594-b107-a770723ee27a service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:00.989494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081183d5-f5e2-47f2-a13e-32a22e1ca014 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:01.018936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 918/1820] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:01.108832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081183d5-f5e2-47f2-a13e-32a22e1ca014 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:01.117602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:01.120351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:01.120763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:01.121157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] PATH_INFO: `/v3/users/f5b34b5b588f4b67b3addfe877854ae6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:01.122983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5b34b5b588f4b67b3addfe877854ae6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:01.125423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:01.243685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5b34b5b588f4b67b3addfe877854ae6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:01.245328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5b34b5b588f4b67b3addfe877854ae6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:01.246354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5b34b5b588f4b67b3addfe877854ae6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:01.274298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5b34b5b588f4b67b3addfe877854ae6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:01.287531 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5b34b5b588f4b67b3addfe877854ae6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:01.299796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:01.338962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 903/1821] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:00 2022] DELETE /identity/v3/users/f5b34b5b588f4b67b3addfe877854ae6 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:01.343126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50eb460-2f4b-439a-8451-a84511936c83 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:01.343419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50eb460-2f4b-439a-8451-a84511936c83 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:01.343695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50eb460-2f4b-439a-8451-a84511936c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:01.371237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-e50eb460-2f4b-439a-8451-a84511936c83 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:01.373045 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e50eb460-2f4b-439a-8451-a84511936c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:01.560315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 919/1822] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:01.653265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:01.655851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:01.778497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50eb460-2f4b-439a-8451-a84511936c83 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:01.781391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e50eb460-2f4b-439a-8451-a84511936c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:01.816130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1d67a-d807-4050-932c-c22dac4a21f5 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:01.822249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856bae21-22b4-4de0-88ed-efcfb86660ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:01.822925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bae21-22b4-4de0-88ed-efcfb86660ec service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:01.823244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bae21-22b4-4de0-88ed-efcfb86660ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:01.823522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856bae21-22b4-4de0-88ed-efcfb86660ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:01.837751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bae21-22b4-4de0-88ed-efcfb86660ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:01.839638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856bae21-22b4-4de0-88ed-efcfb86660ec service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:01.855120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 904/1823] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:01.902765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:01.924311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fd8e4-61a4-4a9e-a2ff-ac9ae12cc927 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:01.924927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fd8e4-61a4-4a9e-a2ff-ac9ae12cc927 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:01.925389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fd8e4-61a4-4a9e-a2ff-ac9ae12cc927 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:01.928336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 905/1824] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:14:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:14:01.988527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50eb460-2f4b-439a-8451-a84511936c83 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:01.997627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b1466-c141-4ef8-b17f-b237d3666f39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:01.999108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b1466-c141-4ef8-b17f-b237d3666f39 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:01.999316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b1466-c141-4ef8-b17f-b237d3666f39 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:01.999720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b1466-c141-4ef8-b17f-b237d3666f39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:02.003541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-dd7b1466-c141-4ef8-b17f-b237d3666f39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:02.023653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04209-fc22-471b-a11b-1a7197eb1cdf None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:02.024696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b1466-c141-4ef8-b17f-b237d3666f39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:02.025020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04209-fc22-471b-a11b-1a7197eb1cdf None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:02.025642 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc04209-fc22-471b-a11b-1a7197eb1cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:02.026657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b1466-c141-4ef8-b17f-b237d3666f39 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:02.053176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 920/1825] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:02.075740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-bdc04209-fc22-471b-a11b-1a7197eb1cdf None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:02.077221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bdc04209-fc22-471b-a11b-1a7197eb1cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:02.106551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 906/1826] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:14:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:02.669125 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b1466-c141-4ef8-b17f-b237d3666f39 service nova] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:02.671108 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-dd7b1466-c141-4ef8-b17f-b237d3666f39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:02.713648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b1466-c141-4ef8-b17f-b237d3666f39 service nova] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:02.717592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc04209-fc22-471b-a11b-1a7197eb1cdf None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:02.719249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bdc04209-fc22-471b-a11b-1a7197eb1cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:02.721170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:02.722147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:02.723200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:02.723558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] PATH_INFO: `/v3/projects/004171b6113d444396ba9b0985462401` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:02.725165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=004171b6113d444396ba9b0985462401)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:02.727254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:02.765935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc04209-fc22-471b-a11b-1a7197eb1cdf None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:02.781242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ed6f0-1a1a-4cca-ae0d-6aaf538955d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:02.781783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ed6f0-1a1a-4cca-ae0d-6aaf538955d5 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:02.782557 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ed6f0-1a1a-4cca-ae0d-6aaf538955d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:02.783501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ed6f0-1a1a-4cca-ae0d-6aaf538955d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:02.795602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ed6f0-1a1a-4cca-ae0d-6aaf538955d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:02.797593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ed6f0-1a1a-4cca-ae0d-6aaf538955d5 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:02.804117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004171b6113d444396ba9b0985462401', 'request_id': 'req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:02.822231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004171b6113d444396ba9b0985462401', 'request_id': 'req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:02.824419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 907/1827] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:02.831433 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004171b6113d444396ba9b0985462401', 'request_id': 'req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:02.845898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] The token cache is being invalidate because project 004171b6113d444396ba9b0985462401 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:02.846199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:02.848191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:02.860657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 921/1828] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:02 2022] DELETE /identity/v3/projects/004171b6113d444396ba9b0985462401 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:02.862244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ed6f0-1a1a-4cca-ae0d-6aaf538955d5 service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:02.875340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ed6f0-1a1a-4cca-ae0d-6aaf538955d5 service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:02.939937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:02.941094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:02.941631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:02.942093 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] PATH_INFO: `/v3/users/9feef3a703b14b1881a6aeff302e5e1d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:02.943572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9feef3a703b14b1881a6aeff302e5e1d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:02.945718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:03.002561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9feef3a703b14b1881a6aeff302e5e1d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.003435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9feef3a703b14b1881a6aeff302e5e1d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.004813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9feef3a703b14b1881a6aeff302e5e1d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.014826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9feef3a703b14b1881a6aeff302e5e1d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.022471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9feef3a703b14b1881a6aeff302e5e1d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.028595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:03.044160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 908/1829] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:02 2022] DELETE /identity/v3/users/9feef3a703b14b1881a6aeff302e5e1d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:03.330677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:03.332942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:03.340419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:03.341910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:03.350980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888d406-d05f-4bc8-a4f8-d6e68f2d96d4 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:03.422559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f489b8a-92c0-437d-94b6-f6ba7fe661c9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:03.423690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:03.424662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:03.424883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:03.425216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] PATH_INFO: `/v3/users/8e7daef95586424b8f812ad5ddcf3657` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:03.426867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e7daef95586424b8f812ad5ddcf3657)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:03.427481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:03.429057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:03.429206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:03.429362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:03.429815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] PATH_INFO: `/v3/projects/8fd0d64ab4d8422f8df93274895d0ec9` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:03.431021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd0d64ab4d8422f8df93274895d0ec9)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:03.433001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:03.488777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:03.490689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e7daef95586424b8f812ad5ddcf3657'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.493029 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e7daef95586424b8f812ad5ddcf3657'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.494177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e7daef95586424b8f812ad5ddcf3657'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.494764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd0d64ab4d8422f8df93274895d0ec9', 'request_id': 'req-6c33a752-44de-4a88-872d-23db5ce786d3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.506154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e7daef95586424b8f812ad5ddcf3657'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.511525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd0d64ab4d8422f8df93274895d0ec9', 'request_id': 'req-6c33a752-44de-4a88-872d-23db5ce786d3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.521085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd0d64ab4d8422f8df93274895d0ec9', 'request_id': 'req-6c33a752-44de-4a88-872d-23db5ce786d3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.523938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2f9cce2-57cc-4db7-9c45-6907e558f2c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e7daef95586424b8f812ad5ddcf3657'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.536977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] The token cache is being invalidate because project 8fd0d64ab4d8422f8df93274895d0ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:03.537283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.538267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6c33a752-44de-4a88-872d-23db5ce786d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:03.539933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 909/1830] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:03 2022] DELETE /identity/v3/projects/8fd0d64ab4d8422f8df93274895d0ec9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:03.546583 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bec8d-4720-4ed7-8460-7eec36a9ece9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:03.546900 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bec8d-4720-4ed7-8460-7eec36a9ece9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:03.548230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053bec8d-4720-4ed7-8460-7eec36a9ece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:03.559210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 922/1831] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:03 2022] DELETE /identity/v3/users/8e7daef95586424b8f812ad5ddcf3657 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:03.567219 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0daa0-86a9-48e7-acf8-45696640a1ff None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:03.567763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0daa0-86a9-48e7-acf8-45696640a1ff None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:03.568115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0daa0-86a9-48e7-acf8-45696640a1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:03.579630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-053bec8d-4720-4ed7-8460-7eec36a9ece9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:03.581691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-053bec8d-4720-4ed7-8460-7eec36a9ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:03.593956 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-32f0daa0-86a9-48e7-acf8-45696640a1ff None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:03.594767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-32f0daa0-86a9-48e7-acf8-45696640a1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:03.673100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 923/1832] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:14:03.686264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0daa0-86a9-48e7-acf8-45696640a1ff None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:03.686468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-32f0daa0-86a9-48e7-acf8-45696640a1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:03.710131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 910/1833] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:14:03.720420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3b6ae-f8ab-413c-bf06-4cfbed885462 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:03.720737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3b6ae-f8ab-413c-bf06-4cfbed885462 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:03.721051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd3b6ae-f8ab-413c-bf06-4cfbed885462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:03.752489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-acd3b6ae-f8ab-413c-bf06-4cfbed885462 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:03.754807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-acd3b6ae-f8ab-413c-bf06-4cfbed885462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:03.771777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0daa0-86a9-48e7-acf8-45696640a1ff None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:03.772290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 911/1834] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:03.776083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab77dd0-b50b-4729-a7ea-855133fa1bbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:03.776754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab77dd0-b50b-4729-a7ea-855133fa1bbb service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:03.777153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab77dd0-b50b-4729-a7ea-855133fa1bbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:03.777505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab77dd0-b50b-4729-a7ea-855133fa1bbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:03.777792 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3b6ae-f8ab-413c-bf06-4cfbed885462 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:03.789250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab77dd0-b50b-4729-a7ea-855133fa1bbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:03.790928 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab77dd0-b50b-4729-a7ea-855133fa1bbb service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:03.792992 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd3b6ae-f8ab-413c-bf06-4cfbed885462 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:03.803784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 924/1835] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:03.811854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60670ed-d5a8-4b24-8816-706ed940606a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:03.812537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60670ed-d5a8-4b24-8816-706ed940606a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:03.812822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60670ed-d5a8-4b24-8816-706ed940606a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:03.813100 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60670ed-d5a8-4b24-8816-706ed940606a admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:03.813839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60670ed-d5a8-4b24-8816-706ed940606a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:03.814151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60670ed-d5a8-4b24-8816-706ed940606a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:03.815932 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60670ed-d5a8-4b24-8816-706ed940606a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:03.824026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:03.830093 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 912/1836] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:14:03.835581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab77dd0-b50b-4729-a7ea-855133fa1bbb service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:03.848068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab77dd0-b50b-4729-a7ea-855133fa1bbb service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:03.852743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52c7b2-eb05-4dd6-b4db-b5f7160d9c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:03.853440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52c7b2-eb05-4dd6-b4db-b5f7160d9c4b admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:03.853755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52c7b2-eb05-4dd6-b4db-b5f7160d9c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:03.854068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52c7b2-eb05-4dd6-b4db-b5f7160d9c4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:03.854820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52c7b2-eb05-4dd6-b4db-b5f7160d9c4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:03.857022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52c7b2-eb05-4dd6-b4db-b5f7160d9c4b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:03.890803 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 925/1837] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:03 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:03.892123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60670ed-d5a8-4b24-8816-706ed940606a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:03.905151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60670ed-d5a8-4b24-8816-706ed940606a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:03.974084 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add30810-9dd0-474d-b10c-530f340baa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:03.974907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add30810-9dd0-474d-b10c-530f340baa4b admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:03.975297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add30810-9dd0-474d-b10c-530f340baa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:03.975680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add30810-9dd0-474d-b10c-530f340baa4b admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:03.976429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add30810-9dd0-474d-b10c-530f340baa4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:03.978505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add30810-9dd0-474d-b10c-530f340baa4b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:04.005579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 913/1838] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:03 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:04.009846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52c7b2-eb05-4dd6-b4db-b5f7160d9c4b admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:04.022113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52c7b2-eb05-4dd6-b4db-b5f7160d9c4b admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:04.028994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35191567-84bc-4ac0-875d-cb33eb1573f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:04.029783 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35191567-84bc-4ac0-875d-cb33eb1573f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:04.030070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35191567-84bc-4ac0-875d-cb33eb1573f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:04.030357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35191567-84bc-4ac0-875d-cb33eb1573f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:04.031136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35191567-84bc-4ac0-875d-cb33eb1573f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:04.032746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35191567-84bc-4ac0-875d-cb33eb1573f8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:04.044580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 926/1839] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:04.049062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add30810-9dd0-474d-b10c-530f340baa4b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:04.064091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add30810-9dd0-474d-b10c-530f340baa4b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:04.077705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda19064-aed8-4d36-a771-baec16f0a4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:04.078793 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda19064-aed8-4d36-a771-baec16f0a4fb admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:04.089256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda19064-aed8-4d36-a771-baec16f0a4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:04.089553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda19064-aed8-4d36-a771-baec16f0a4fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:04.090401 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda19064-aed8-4d36-a771-baec16f0a4fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:04.092253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda19064-aed8-4d36-a771-baec16f0a4fb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:04.111090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 914/1840] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:04.116156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35191567-84bc-4ac0-875d-cb33eb1573f8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:04.127634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35191567-84bc-4ac0-875d-cb33eb1573f8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:04.132915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432531be-6115-4c3f-b1de-3b517628d067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:04.133819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432531be-6115-4c3f-b1de-3b517628d067 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:04.134085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432531be-6115-4c3f-b1de-3b517628d067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:04.134363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432531be-6115-4c3f-b1de-3b517628d067 admin admin] PATH_INFO: `/v3/projects/8ca5af1b59244f798024cbda56f7e8fc/users/b2ecddfd823049659ebdd10facf493ed/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:04.154483 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432531be-6115-4c3f-b1de-3b517628d067 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ca5af1b59244f798024cbda56f7e8fc, user_id=b2ecddfd823049659ebdd10facf493ed, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:04.158317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432531be-6115-4c3f-b1de-3b517628d067 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:04.169851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda19064-aed8-4d36-a771-baec16f0a4fb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:04.171728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fda19064-aed8-4d36-a771-baec16f0a4fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:04.179932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 927/1841] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:04 2022] PUT /identity/v3/projects/8ca5af1b59244f798024cbda56f7e8fc/users/b2ecddfd823049659ebdd10facf493ed/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:04.192032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71486e76-3a66-48d8-ad3d-8215cbb6d019 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:04.192676 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71486e76-3a66-48d8-ad3d-8215cbb6d019 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:04.193160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71486e76-3a66-48d8-ad3d-8215cbb6d019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:04.201771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda19064-aed8-4d36-a771-baec16f0a4fb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:04.217511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:04.237653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-71486e76-3a66-48d8-ad3d-8215cbb6d019 None None] MFA Rules not processed for user `b2ecddfd823049659ebdd10facf493ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:04.238998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-71486e76-3a66-48d8-ad3d-8215cbb6d019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:04.282957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:04.283896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:04.284185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:04.284465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] PATH_INFO: `/v3/users/625f75bf4b604d79ab3af239da62fbbc` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:04.286248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=625f75bf4b604d79ab3af239da62fbbc)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:04.288721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:04.327109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 928/1842] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:14:04.340308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71486e76-3a66-48d8-ad3d-8215cbb6d019 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:04.342262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-71486e76-3a66-48d8-ad3d-8215cbb6d019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:04.349404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '625f75bf4b604d79ab3af239da62fbbc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.350180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '625f75bf4b604d79ab3af239da62fbbc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.350911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '625f75bf4b604d79ab3af239da62fbbc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.360820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '625f75bf4b604d79ab3af239da62fbbc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.370273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-237ae541-3e96-417f-9ab1-46a20c5d0cbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '625f75bf4b604d79ab3af239da62fbbc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.385068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:04.393855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 915/1843] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:04 2022] DELETE /identity/v3/users/625f75bf4b604d79ab3af239da62fbbc => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:04.398748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf39ef6-6e45-45ab-a636-e849b5ce7dfb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:04.399287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf39ef6-6e45-45ab-a636-e849b5ce7dfb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:04.399688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf39ef6-6e45-45ab-a636-e849b5ce7dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:04.435224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71486e76-3a66-48d8-ad3d-8215cbb6d019 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:04.436866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2bf39ef6-6e45-45ab-a636-e849b5ce7dfb None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:04.438088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2bf39ef6-6e45-45ab-a636-e849b5ce7dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:04.511083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c1cef-fff2-4c04-a67e-c070295346f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450bcd0f22b840c299d08ce9433d6c85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:04.511751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c1cef-fff2-4c04-a67e-c070295346f5 service glance] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:04.512033 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c1cef-fff2-4c04-a67e-c070295346f5 service glance] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:04.512308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c1cef-fff2-4c04-a67e-c070295346f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:04.514104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b36c1cef-fff2-4c04-a67e-c070295346f5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:04.546639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 916/1844] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:04.547541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf39ef6-6e45-45ab-a636-e849b5ce7dfb None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:04.548861 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2bf39ef6-6e45-45ab-a636-e849b5ce7dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:04.560912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf39ef6-6e45-45ab-a636-e849b5ce7dfb None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:04.565663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:04.566550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:04.566857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:04.567193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] PATH_INFO: `/v3/projects/bca19d45972e4eb692a8a74156278e3b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:04.568387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca19d45972e4eb692a8a74156278e3b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:04.570308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:04.575401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c1cef-fff2-4c04-a67e-c070295346f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:04.576835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c1cef-fff2-4c04-a67e-c070295346f5 service glance] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:04.590201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 929/1845] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:04.591259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c1cef-fff2-4c04-a67e-c070295346f5 service glance] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:04.592554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b36c1cef-fff2-4c04-a67e-c070295346f5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:04.604156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:04.630800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca19d45972e4eb692a8a74156278e3b', 'request_id': 'req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.640716 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca19d45972e4eb692a8a74156278e3b', 'request_id': 'req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.647917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca19d45972e4eb692a8a74156278e3b', 'request_id': 'req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.661032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c1cef-fff2-4c04-a67e-c070295346f5 service glance] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:04.661156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] The token cache is being invalidate because project bca19d45972e4eb692a8a74156278e3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:04.661558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.662333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.664305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 917/1846] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:04 2022] DELETE /identity/v3/projects/bca19d45972e4eb692a8a74156278e3b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:04.668193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:04.669741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:04.730518 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c81abf-6405-4e82-a899-a38bc2892178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:04.731243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c81abf-6405-4e82-a899-a38bc2892178 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:04.731533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c81abf-6405-4e82-a899-a38bc2892178 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:04.732002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c81abf-6405-4e82-a899-a38bc2892178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:04.733990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-84c81abf-6405-4e82-a899-a38bc2892178 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:04.760896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbb2b0-53f2-49fb-b19d-4cc448a9ca8e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:04.769311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:04.770358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:04.771061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:04.771384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] PATH_INFO: `/v3/users/ba19da460f7d45cc985d23171efc51f5` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:04.773447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba19da460f7d45cc985d23171efc51f5)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:04.775581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:04.807622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:04.818904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c81abf-6405-4e82-a899-a38bc2892178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:04.820715 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c81abf-6405-4e82-a899-a38bc2892178 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:04.833714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 930/1847] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:04.853658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba19da460f7d45cc985d23171efc51f5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.854355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba19da460f7d45cc985d23171efc51f5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.855229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba19da460f7d45cc985d23171efc51f5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.867447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba19da460f7d45cc985d23171efc51f5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.880675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba19da460f7d45cc985d23171efc51f5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:04.914387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 918/1848] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:04 2022] DELETE /identity/v3/users/ba19da460f7d45cc985d23171efc51f5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:05.558599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c81abf-6405-4e82-a899-a38bc2892178 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:05.599689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c81abf-6405-4e82-a899-a38bc2892178 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:05.665347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:05.666983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:05.706660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:05.708357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:05.710098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:05.710461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] PATH_INFO: `/v3/projects/ddfd5b17b3ef4e99b880599b7403ee4a` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:05.712101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddfd5b17b3ef4e99b880599b7403ee4a)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:05.714099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:05.739285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:05.757595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e97d96d-7b67-4e63-b5e5-73f21d75087e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:05.764175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:05.765140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:05.765430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:05.765712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] PATH_INFO: `/v3/projects/98eff08c2639423cb6b73ee9035e3818` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:05.766990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] RBAC: Authorizing `identity:delete_project(project_id=98eff08c2639423cb6b73ee9035e3818)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:05.769224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:05.804586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddfd5b17b3ef4e99b880599b7403ee4a', 'request_id': 'req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:05.812831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddfd5b17b3ef4e99b880599b7403ee4a', 'request_id': 'req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:05.820459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddfd5b17b3ef4e99b880599b7403ee4a', 'request_id': 'req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:05.823944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98eff08c2639423cb6b73ee9035e3818', 'request_id': 'req-c440c629-c37c-4230-b87b-10f3117e637d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:05.831636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98eff08c2639423cb6b73ee9035e3818', 'request_id': 'req-c440c629-c37c-4230-b87b-10f3117e637d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:05.831961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] The token cache is being invalidate because project ddfd5b17b3ef4e99b880599b7403ee4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:05.832300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:05.832988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c2780bd2-fc25-4e5d-babf-e44d8fc1de9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:05.834538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 931/1849] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:05 2022] DELETE /identity/v3/projects/ddfd5b17b3ef4e99b880599b7403ee4a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:05.842537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98eff08c2639423cb6b73ee9035e3818', 'request_id': 'req-c440c629-c37c-4230-b87b-10f3117e637d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:05.843475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400e33e-6579-4091-91ba-d8c917ea8693 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:05.843772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400e33e-6579-4091-91ba-d8c917ea8693 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:05.844058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400e33e-6579-4091-91ba-d8c917ea8693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:05.853263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] The token cache is being invalidate because project 98eff08c2639423cb6b73ee9035e3818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:05.853557 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:05.854223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:05.855775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 919/1850] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:05 2022] DELETE /identity/v3/projects/98eff08c2639423cb6b73ee9035e3818 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:05.859491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:05.860797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:05.862028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-7400e33e-6579-4091-91ba-d8c917ea8693 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:05.863739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7400e33e-6579-4091-91ba-d8c917ea8693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:05.894462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:05.922116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 932/1851] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:14:05.930170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010ba6b-a02b-42bb-8cd5-85836780c634 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:05.930323 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440c629-c37c-4230-b87b-10f3117e637d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:05.930516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010ba6b-a02b-42bb-8cd5-85836780c634 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:05.930741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010ba6b-a02b-42bb-8cd5-85836780c634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:05.935173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:05.936046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:05.936332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:05.936607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] PATH_INFO: `/v3/users/687f98967c844f5da958dea2fae17671` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:05.937910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=687f98967c844f5da958dea2fae17671)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:05.939869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:05.950173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5010ba6b-a02b-42bb-8cd5-85836780c634 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:05.951422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5010ba6b-a02b-42bb-8cd5-85836780c634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:05.962040 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 933/1852] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:14:05.965993 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010ba6b-a02b-42bb-8cd5-85836780c634 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:05.975305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010ba6b-a02b-42bb-8cd5-85836780c634 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:05.983847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4232ead-52e5-4635-9e73-f8df1a1f33e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:05.987299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4232ead-52e5-4635-9e73-f8df1a1f33e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:05.987627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4232ead-52e5-4635-9e73-f8df1a1f33e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:05.988460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4232ead-52e5-4635-9e73-f8df1a1f33e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:05.989218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4232ead-52e5-4635-9e73-f8df1a1f33e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:05.989539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4232ead-52e5-4635-9e73-f8df1a1f33e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:05.991413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4232ead-52e5-4635-9e73-f8df1a1f33e3 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.011209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 934/1853] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:14:06.016507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4232ead-52e5-4635-9e73-f8df1a1f33e3 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.019921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '687f98967c844f5da958dea2fae17671'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.020877 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '687f98967c844f5da958dea2fae17671'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.022141 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '687f98967c844f5da958dea2fae17671'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.028638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4232ead-52e5-4635-9e73-f8df1a1f33e3 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.035020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c03f5-4fd5-4964-8ebe-02d358975c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.036052 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c03f5-4fd5-4964-8ebe-02d358975c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.036349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c03f5-4fd5-4964-8ebe-02d358975c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.036631 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c03f5-4fd5-4964-8ebe-02d358975c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.037457 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c03f5-4fd5-4964-8ebe-02d358975c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.039664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c03f5-4fd5-4964-8ebe-02d358975c33 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.045295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '687f98967c844f5da958dea2fae17671'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.053561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '687f98967c844f5da958dea2fae17671'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.067987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 935/1854] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:06 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:06.072280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c03f5-4fd5-4964-8ebe-02d358975c33 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.072632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 920/1855] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:05 2022] DELETE /identity/v3/users/687f98967c844f5da958dea2fae17671 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:06.074079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d52c03f5-4fd5-4964-8ebe-02d358975c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:06.149202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c03f5-4fd5-4964-8ebe-02d358975c33 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.159013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:06.223770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb5244-cf50-4b64-9161-46348343c297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.224765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb5244-cf50-4b64-9161-46348343c297 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.225050 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb5244-cf50-4b64-9161-46348343c297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.225326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb5244-cf50-4b64-9161-46348343c297 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.226042 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb5244-cf50-4b64-9161-46348343c297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.227974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb5244-cf50-4b64-9161-46348343c297 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.257763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 936/1856] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:06 2022] POST /identity/v3/users => generated 287 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:06.262742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.273489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd256e-9db8-4c14-90c1-c5d94c601790 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.278223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a285a-10ab-4ce9-ae8c-95f112ceb5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.278998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a285a-10ab-4ce9-ae8c-95f112ceb5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.279384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a285a-10ab-4ce9-ae8c-95f112ceb5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.279739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a285a-10ab-4ce9-ae8c-95f112ceb5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.280484 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a285a-10ab-4ce9-ae8c-95f112ceb5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.282101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a285a-10ab-4ce9-ae8c-95f112ceb5b4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.293465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 921/1857] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:06.298014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb5244-cf50-4b64-9161-46348343c297 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.310881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb5244-cf50-4b64-9161-46348343c297 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.315640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d957b34-f9c8-4609-9f33-d6b09d30d393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.316963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d957b34-f9c8-4609-9f33-d6b09d30d393 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.317364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d957b34-f9c8-4609-9f33-d6b09d30d393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.317671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d957b34-f9c8-4609-9f33-d6b09d30d393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.318619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d957b34-f9c8-4609-9f33-d6b09d30d393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.323908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d957b34-f9c8-4609-9f33-d6b09d30d393 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.343408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 937/1858] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:06.344724 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a285a-10ab-4ce9-ae8c-95f112ceb5b4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.355487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a285a-10ab-4ce9-ae8c-95f112ceb5b4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.362110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1f7a0-3ef6-46e1-be95-0d9e942feb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.363097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1f7a0-3ef6-46e1-be95-0d9e942feb17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.363388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1f7a0-3ef6-46e1-be95-0d9e942feb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.363677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1f7a0-3ef6-46e1-be95-0d9e942feb17 admin admin] PATH_INFO: `/v3/projects/808f3e9aaa2e47a1b34a9a9e48101c0e/users/c13306f9af6343549b45fa5960e84c54/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.381887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1f7a0-3ef6-46e1-be95-0d9e942feb17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=808f3e9aaa2e47a1b34a9a9e48101c0e, user_id=c13306f9af6343549b45fa5960e84c54, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.385452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1f7a0-3ef6-46e1-be95-0d9e942feb17 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.398536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d957b34-f9c8-4609-9f33-d6b09d30d393 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.401058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5d957b34-f9c8-4609-9f33-d6b09d30d393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:06.403892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 922/1859] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:06 2022] PUT /identity/v3/projects/808f3e9aaa2e47a1b34a9a9e48101c0e/users/c13306f9af6343549b45fa5960e84c54/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:06.419215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510893d7-1ec3-45df-ace8-35a7f85d72b1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.419215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510893d7-1ec3-45df-ace8-35a7f85d72b1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.419215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510893d7-1ec3-45df-ace8-35a7f85d72b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.438184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d957b34-f9c8-4609-9f33-d6b09d30d393 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.459252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-510893d7-1ec3-45df-ace8-35a7f85d72b1 None None] MFA Rules not processed for user `c13306f9af6343549b45fa5960e84c54`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:06.460675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-510893d7-1ec3-45df-ace8-35a7f85d72b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:06.521847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.522787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.523269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.523393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] PATH_INFO: `/v3/projects/387c2b2715af4903bab3420b36e54681` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.524602 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387c2b2715af4903bab3420b36e54681)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.526508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.543596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 923/1860] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:14:06.547983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510893d7-1ec3-45df-ace8-35a7f85d72b1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.557544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:06.568432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510893d7-1ec3-45df-ace8-35a7f85d72b1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.579351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31bdd5-cd10-4f2e-a95f-a11bbc767fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.581861 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31bdd5-cd10-4f2e-a95f-a11bbc767fda admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.582327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31bdd5-cd10-4f2e-a95f-a11bbc767fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.582829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31bdd5-cd10-4f2e-a95f-a11bbc767fda admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.584393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31bdd5-cd10-4f2e-a95f-a11bbc767fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.587908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31bdd5-cd10-4f2e-a95f-a11bbc767fda admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.592317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387c2b2715af4903bab3420b36e54681', 'request_id': 'req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.603445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387c2b2715af4903bab3420b36e54681', 'request_id': 'req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.610523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387c2b2715af4903bab3420b36e54681', 'request_id': 'req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.626311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] The token cache is being invalidate because project 387c2b2715af4903bab3420b36e54681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:06.626682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.627530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.630055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 938/1861] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:06 2022] DELETE /identity/v3/projects/387c2b2715af4903bab3420b36e54681 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:06.633626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.634961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:06.635988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 924/1862] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:06 2022] POST /identity/v3/projects => generated 371 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:06.639632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31bdd5-cd10-4f2e-a95f-a11bbc767fda admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.642489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-db31bdd5-cd10-4f2e-a95f-a11bbc767fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:06.659091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:06.713017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb3b70-36af-4210-8fbd-b8e0e21c49f4 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.726306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31bdd5-cd10-4f2e-a95f-a11bbc767fda admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.730989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb895-2a17-42f2-b8a6-a7ca13691118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.731745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb895-2a17-42f2-b8a6-a7ca13691118 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.732075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb895-2a17-42f2-b8a6-a7ca13691118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.732342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcb895-2a17-42f2-b8a6-a7ca13691118 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.733040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcb895-2a17-42f2-b8a6-a7ca13691118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.735082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcb895-2a17-42f2-b8a6-a7ca13691118 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.762157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 925/1863] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:06 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:06.765949 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb895-2a17-42f2-b8a6-a7ca13691118 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.776349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcb895-2a17-42f2-b8a6-a7ca13691118 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.781472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c9203-05c0-4627-9c1d-90b9ccfb8258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.782146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c9203-05c0-4627-9c1d-90b9ccfb8258 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.782436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c9203-05c0-4627-9c1d-90b9ccfb8258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.782726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c9203-05c0-4627-9c1d-90b9ccfb8258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.783494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c9203-05c0-4627-9c1d-90b9ccfb8258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.785147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c9203-05c0-4627-9c1d-90b9ccfb8258 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.787343 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.788485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.788777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.789083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] PATH_INFO: `/v3/users/4249f597089b417aa42c40263fec9ebc` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.790416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4249f597089b417aa42c40263fec9ebc)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.792450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.802670 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 926/1864] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:14:06.803685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c9203-05c0-4627-9c1d-90b9ccfb8258 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.829291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c9203-05c0-4627-9c1d-90b9ccfb8258 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:06.834859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037370d8-9953-4181-a4fa-0e7a409d3c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:06.836021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037370d8-9953-4181-a4fa-0e7a409d3c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:06.836313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037370d8-9953-4181-a4fa-0e7a409d3c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:06.837055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037370d8-9953-4181-a4fa-0e7a409d3c57 admin admin] PATH_INFO: `/v3/projects/f62fd70c29084d6b9a8a6c872c0f6236/users/53b3618ae3174904bdfa4dc823506ef1/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:06.859705 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037370d8-9953-4181-a4fa-0e7a409d3c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62fd70c29084d6b9a8a6c872c0f6236, user_id=53b3618ae3174904bdfa4dc823506ef1, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:06.862886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037370d8-9953-4181-a4fa-0e7a409d3c57 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:06.864681 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4249f597089b417aa42c40263fec9ebc'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.865428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4249f597089b417aa42c40263fec9ebc'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.866062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4249f597089b417aa42c40263fec9ebc'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.874444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4249f597089b417aa42c40263fec9ebc'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.881761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 927/1865] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:06 2022] PUT /identity/v3/projects/f62fd70c29084d6b9a8a6c872c0f6236/users/53b3618ae3174904bdfa4dc823506ef1/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:06.886740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037370d8-9953-4181-a4fa-0e7a409d3c57 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:06.888135 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4249f597089b417aa42c40263fec9ebc'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:06.889004 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-037370d8-9953-4181-a4fa-0e7a409d3c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:06.920379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 939/1866] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:06 2022] DELETE /identity/v3/users/4249f597089b417aa42c40263fec9ebc => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:06.963381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037370d8-9953-4181-a4fa-0e7a409d3c57 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:07.019536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:07.041731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa8de0-7496-47b5-a34e-30124acf292a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:07.043020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa8de0-7496-47b5-a34e-30124acf292a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:07.043374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa8de0-7496-47b5-a34e-30124acf292a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:07.043825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa8de0-7496-47b5-a34e-30124acf292a admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:07.044603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa8de0-7496-47b5-a34e-30124acf292a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:07.046479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa8de0-7496-47b5-a34e-30124acf292a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:07.064860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 928/1867] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:14:07.066752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:07.078464 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05339a24-4cc9-4716-b256-1d9bab9a07e5 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:07.083307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1653c-dcaa-4bbb-b2a3-b1f178203066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:07.084203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1653c-dcaa-4bbb-b2a3-b1f178203066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:07.084485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1653c-dcaa-4bbb-b2a3-b1f178203066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:07.084760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1653c-dcaa-4bbb-b2a3-b1f178203066 admin admin] PATH_INFO: `/v3/projects/f62fd70c29084d6b9a8a6c872c0f6236/users/53b3618ae3174904bdfa4dc823506ef1/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:07.087248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1653c-dcaa-4bbb-b2a3-b1f178203066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62fd70c29084d6b9a8a6c872c0f6236, user_id=53b3618ae3174904bdfa4dc823506ef1, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:07.089977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1653c-dcaa-4bbb-b2a3-b1f178203066 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:07.110054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 940/1868] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:07 2022] PUT /identity/v3/projects/f62fd70c29084d6b9a8a6c872c0f6236/users/53b3618ae3174904bdfa4dc823506ef1/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:07.122282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893712b4-6cff-4232-a9af-22f702bcfee5 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:07.122574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893712b4-6cff-4232-a9af-22f702bcfee5 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:07.122854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893712b4-6cff-4232-a9af-22f702bcfee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:07.149168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-893712b4-6cff-4232-a9af-22f702bcfee5 None None] MFA Rules not processed for user `53b3618ae3174904bdfa4dc823506ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:07.150411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-893712b4-6cff-4232-a9af-22f702bcfee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:07.232213 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 929/1869] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:07 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:07.246097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1653c-dcaa-4bbb-b2a3-b1f178203066 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:07.248137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-94b1653c-dcaa-4bbb-b2a3-b1f178203066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:07.255412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893712b4-6cff-4232-a9af-22f702bcfee5 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:07.257125 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-893712b4-6cff-4232-a9af-22f702bcfee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:07.289611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:07.315617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1653c-dcaa-4bbb-b2a3-b1f178203066 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:07.319332 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22da694-834f-468e-adac-4af767c43e12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:07.319979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22da694-834f-468e-adac-4af767c43e12 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:07.320264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22da694-834f-468e-adac-4af767c43e12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:07.320541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22da694-834f-468e-adac-4af767c43e12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:07.330684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893712b4-6cff-4232-a9af-22f702bcfee5 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:07.333126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22da694-834f-468e-adac-4af767c43e12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:07.334629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22da694-834f-468e-adac-4af767c43e12 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:07.335860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:07.336999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:07.337285 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:07.337563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] PATH_INFO: `/v3/projects/312ba9e3509845a8a9cc2e0b5f532760` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:07.338749 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=312ba9e3509845a8a9cc2e0b5f532760)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:07.340711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:07.345959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 941/1870] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:07 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:07.405209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '312ba9e3509845a8a9cc2e0b5f532760', 'request_id': 'req-77035115-d9de-4726-827a-fa6ddb20eec1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:07.423988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '312ba9e3509845a8a9cc2e0b5f532760', 'request_id': 'req-77035115-d9de-4726-827a-fa6ddb20eec1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:07.431752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '312ba9e3509845a8a9cc2e0b5f532760', 'request_id': 'req-77035115-d9de-4726-827a-fa6ddb20eec1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:07.436860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:07.457850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] The token cache is being invalidate because project 312ba9e3509845a8a9cc2e0b5f532760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:07.457850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:07.459643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-77035115-d9de-4726-827a-fa6ddb20eec1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:07.461537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 930/1871] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:07 2022] DELETE /identity/v3/projects/312ba9e3509845a8a9cc2e0b5f532760 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:07.467804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fca42-0177-4cad-bf65-e143596eef85 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:07.468113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fca42-0177-4cad-bf65-e143596eef85 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:07.468408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68fca42-0177-4cad-bf65-e143596eef85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:07.516876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c68fca42-0177-4cad-bf65-e143596eef85 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:07.518132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c68fca42-0177-4cad-bf65-e143596eef85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:07.657806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 942/1872] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:07.668261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2360f2-be65-4b61-9026-b5d9785d5524 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:07.668954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2360f2-be65-4b61-9026-b5d9785d5524 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:07.669592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2360f2-be65-4b61-9026-b5d9785d5524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:07.718447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ee2360f2-be65-4b61-9026-b5d9785d5524 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:07.720604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ee2360f2-be65-4b61-9026-b5d9785d5524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:07.742111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 931/1873] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:07.745886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fca42-0177-4cad-bf65-e143596eef85 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:07.759220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68fca42-0177-4cad-bf65-e143596eef85 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:07.764289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a5fa4-713f-4530-8b69-8b139d7573de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:07.765049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a5fa4-713f-4530-8b69-8b139d7573de admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:07.765406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a5fa4-713f-4530-8b69-8b139d7573de admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:07.765767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a5fa4-713f-4530-8b69-8b139d7573de admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:07.768060 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a5fa4-713f-4530-8b69-8b139d7573de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:07.768844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a5fa4-713f-4530-8b69-8b139d7573de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:07.786445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a5fa4-713f-4530-8b69-8b139d7573de admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:07.807232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 943/1874] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:14:07.811342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2360f2-be65-4b61-9026-b5d9785d5524 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:07.829993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2360f2-be65-4b61-9026-b5d9785d5524 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:07.847771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8dba8-d4b2-45b9-888d-11c0a04b5d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:07.848459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8dba8-d4b2-45b9-888d-11c0a04b5d58 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:07.848745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8dba8-d4b2-45b9-888d-11c0a04b5d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:07.849025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8dba8-d4b2-45b9-888d-11c0a04b5d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:07.849734 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8dba8-d4b2-45b9-888d-11c0a04b5d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:07.851764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd8dba8-d4b2-45b9-888d-11c0a04b5d58 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:07.922783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 932/1875] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:07 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:07.927664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a5fa4-713f-4530-8b69-8b139d7573de admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:07.940217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a5fa4-713f-4530-8b69-8b139d7573de admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.022789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec732c-c37e-4597-9461-8a3cd37d0bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.022789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec732c-c37e-4597-9461-8a3cd37d0bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.022789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec732c-c37e-4597-9461-8a3cd37d0bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.022789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec732c-c37e-4597-9461-8a3cd37d0bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.023377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec732c-c37e-4597-9461-8a3cd37d0bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:08.025520 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec732c-c37e-4597-9461-8a3cd37d0bf7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:08.067843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 944/1876] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:07 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:08.071760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8dba8-d4b2-45b9-888d-11c0a04b5d58 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:08.086133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec732c-c37e-4597-9461-8a3cd37d0bf7 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:08.090899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c1ec732c-c37e-4597-9461-8a3cd37d0bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:08.092826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8dba8-d4b2-45b9-888d-11c0a04b5d58 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.106826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec732c-c37e-4597-9461-8a3cd37d0bf7 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.112514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42326f9-65e4-41ef-89c6-2fed3afc0235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.112514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42326f9-65e4-41ef-89c6-2fed3afc0235 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.112514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42326f9-65e4-41ef-89c6-2fed3afc0235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.113415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42326f9-65e4-41ef-89c6-2fed3afc0235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.113916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42326f9-65e4-41ef-89c6-2fed3afc0235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:08.115558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42326f9-65e4-41ef-89c6-2fed3afc0235 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:08.116322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.117318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.117702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.118151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] PATH_INFO: `/v3/users/c13306f9af6343549b45fa5960e84c54` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.119606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13306f9af6343549b45fa5960e84c54)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:08.121819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:08.143669 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 933/1877] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:08.147853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42326f9-65e4-41ef-89c6-2fed3afc0235 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:08.164860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42326f9-65e4-41ef-89c6-2fed3afc0235 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.173492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d45a8-3911-433c-9a1a-ac8374e08fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.174113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d45a8-3911-433c-9a1a-ac8374e08fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.174355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d45a8-3911-433c-9a1a-ac8374e08fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.174778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d45a8-3911-433c-9a1a-ac8374e08fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.176114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d45a8-3911-433c-9a1a-ac8374e08fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:08.177804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d45a8-3911-433c-9a1a-ac8374e08fc2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:08.192400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 934/1878] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:08.198022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d45a8-3911-433c-9a1a-ac8374e08fc2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:08.212171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d45a8-3911-433c-9a1a-ac8374e08fc2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.217492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb15ed35-dffd-4fd2-b8f9-77146e4b7694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.218500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15ed35-dffd-4fd2-b8f9-77146e4b7694 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.218673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15ed35-dffd-4fd2-b8f9-77146e4b7694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.218962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb15ed35-dffd-4fd2-b8f9-77146e4b7694 admin admin] PATH_INFO: `/v3/projects/a18639598aa0484aaa6ec6d4470ec2c6/users/2c699a701d1243b98d84ef6f95ea619f/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.229453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13306f9af6343549b45fa5960e84c54'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:08.234171 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13306f9af6343549b45fa5960e84c54'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:08.236515 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13306f9af6343549b45fa5960e84c54'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:08.239458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:08.246594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15ed35-dffd-4fd2-b8f9-77146e4b7694 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a18639598aa0484aaa6ec6d4470ec2c6, user_id=2c699a701d1243b98d84ef6f95ea619f, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:08.249524 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb15ed35-dffd-4fd2-b8f9-77146e4b7694 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:08.260643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13306f9af6343549b45fa5960e84c54'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:08.281721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 935/1879] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:08 2022] PUT /identity/v3/projects/a18639598aa0484aaa6ec6d4470ec2c6/users/2c699a701d1243b98d84ef6f95ea619f/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:08.285949 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952e608-468d-4264-9f67-7ec2e59e91ef None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.286353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952e608-468d-4264-9f67-7ec2e59e91ef None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.286784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5952e608-468d-4264-9f67-7ec2e59e91ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.287809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5aefe557-c442-486e-bdb2-4cff09bf044f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13306f9af6343549b45fa5960e84c54'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:08.324743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5952e608-468d-4264-9f67-7ec2e59e91ef None None] MFA Rules not processed for user `2c699a701d1243b98d84ef6f95ea619f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:08.325983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5952e608-468d-4264-9f67-7ec2e59e91ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:08.327864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 945/1880] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:08 2022] DELETE /identity/v3/users/c13306f9af6343549b45fa5960e84c54 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:08.329528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6160ee6-5c41-4a09-b53b-4957f0f7c2af None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.332243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6160ee6-5c41-4a09-b53b-4957f0f7c2af None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.337744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6160ee6-5c41-4a09-b53b-4957f0f7c2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.372503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-f6160ee6-5c41-4a09-b53b-4957f0f7c2af None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:08.374243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f6160ee6-5c41-4a09-b53b-4957f0f7c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:08.422178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 936/1881] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:08.423770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952e608-468d-4264-9f67-7ec2e59e91ef None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:08.426037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5952e608-468d-4264-9f67-7ec2e59e91ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:08.500926 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 946/1882] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:08.524138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6160ee6-5c41-4a09-b53b-4957f0f7c2af None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:08.525756 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f6160ee6-5c41-4a09-b53b-4957f0f7c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:08.539706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5952e608-468d-4264-9f67-7ec2e59e91ef None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.548931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7658da-2dfc-48fc-a760-da06626a775a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.550012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7658da-2dfc-48fc-a760-da06626a775a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.550511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7658da-2dfc-48fc-a760-da06626a775a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.551355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7658da-2dfc-48fc-a760-da06626a775a admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.552246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7658da-2dfc-48fc-a760-da06626a775a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:08.555819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7658da-2dfc-48fc-a760-da06626a775a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:08.574717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:08.600929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 937/1883] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:08 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:08.605248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7658da-2dfc-48fc-a760-da06626a775a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:08.651306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6160ee6-5c41-4a09-b53b-4957f0f7c2af None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.719746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7658da-2dfc-48fc-a760-da06626a775a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.732944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f59f0a-2ada-442a-aac7-6a6677248fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.733823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f59f0a-2ada-442a-aac7-6a6677248fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.734373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f59f0a-2ada-442a-aac7-6a6677248fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.734796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f59f0a-2ada-442a-aac7-6a6677248fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.735539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f59f0a-2ada-442a-aac7-6a6677248fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:08.737523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f59f0a-2ada-442a-aac7-6a6677248fb4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:08.745439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04a0b1-b28a-4dfc-9376-9ec754b3028f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.746105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04a0b1-b28a-4dfc-9376-9ec754b3028f service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.746397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04a0b1-b28a-4dfc-9376-9ec754b3028f service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.746680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04a0b1-b28a-4dfc-9376-9ec754b3028f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.760172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04a0b1-b28a-4dfc-9376-9ec754b3028f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:08.761712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04a0b1-b28a-4dfc-9376-9ec754b3028f service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:08.762287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 938/1884] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:08 2022] POST /identity/v3/users => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:08.766178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f59f0a-2ada-442a-aac7-6a6677248fb4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:08.774006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 947/1885] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:08.778714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f59f0a-2ada-442a-aac7-6a6677248fb4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.784566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a669933-19c9-4fcd-9539-f27fcaf5e440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.785467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a669933-19c9-4fcd-9539-f27fcaf5e440 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.785883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a669933-19c9-4fcd-9539-f27fcaf5e440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.786289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a669933-19c9-4fcd-9539-f27fcaf5e440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.788001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a669933-19c9-4fcd-9539-f27fcaf5e440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:08.789102 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a669933-19c9-4fcd-9539-f27fcaf5e440 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:08.819937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 939/1886] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:08.821264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04a0b1-b28a-4dfc-9376-9ec754b3028f service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:08.833156 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04a0b1-b28a-4dfc-9376-9ec754b3028f service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.840871 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe714074-e5c7-4cc4-8a99-2d7de0add502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.843848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe714074-e5c7-4cc4-8a99-2d7de0add502 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.844137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe714074-e5c7-4cc4-8a99-2d7de0add502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.844423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe714074-e5c7-4cc4-8a99-2d7de0add502 admin admin] PATH_INFO: `/v3/projects/699ed244e070471492a4d749bdad661b/users/b0de31046c1c4b72a3f32e3a36444e6f/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.867296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe714074-e5c7-4cc4-8a99-2d7de0add502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=699ed244e070471492a4d749bdad661b, user_id=b0de31046c1c4b72a3f32e3a36444e6f, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:08.870866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe714074-e5c7-4cc4-8a99-2d7de0add502 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:08.885860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:08.893650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 948/1887] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:08 2022] PUT /identity/v3/projects/699ed244e070471492a4d749bdad661b/users/b0de31046c1c4b72a3f32e3a36444e6f/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:08.899151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a669933-19c9-4fcd-9539-f27fcaf5e440 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:08.911116 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a669933-19c9-4fcd-9539-f27fcaf5e440 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:08.993808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4672e20-2ecc-4a46-9f91-e037c6184c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:08.994665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4672e20-2ecc-4a46-9f91-e037c6184c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:08.995146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4672e20-2ecc-4a46-9f91-e037c6184c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:08.995608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4672e20-2ecc-4a46-9f91-e037c6184c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:08.996664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4672e20-2ecc-4a46-9f91-e037c6184c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:09.000891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4672e20-2ecc-4a46-9f91-e037c6184c5a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:09.012009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 940/1888] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:09.016247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe714074-e5c7-4cc4-8a99-2d7de0add502 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:09.033061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe714074-e5c7-4cc4-8a99-2d7de0add502 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:09.040762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d321ac3-fd35-4446-805c-1f232b55a0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:09.041748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321ac3-fd35-4446-805c-1f232b55a0fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:09.042034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321ac3-fd35-4446-805c-1f232b55a0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:09.044840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d321ac3-fd35-4446-805c-1f232b55a0fe admin admin] PATH_INFO: `/v3/projects/699ed244e070471492a4d749bdad661b/users/b0de31046c1c4b72a3f32e3a36444e6f/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:09.046442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d321ac3-fd35-4446-805c-1f232b55a0fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=699ed244e070471492a4d749bdad661b, user_id=b0de31046c1c4b72a3f32e3a36444e6f, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:09.049265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d321ac3-fd35-4446-805c-1f232b55a0fe admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:09.063473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 949/1889] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:09 2022] PUT /identity/v3/projects/699ed244e070471492a4d749bdad661b/users/b0de31046c1c4b72a3f32e3a36444e6f/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:09.075109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d736f13-5ab9-413a-a0fd-1471bc40ca17 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:09.075568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d736f13-5ab9-413a-a0fd-1471bc40ca17 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:09.075928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d736f13-5ab9-413a-a0fd-1471bc40ca17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:09.109455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2d736f13-5ab9-413a-a0fd-1471bc40ca17 None None] MFA Rules not processed for user `b0de31046c1c4b72a3f32e3a36444e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:09.110825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2d736f13-5ab9-413a-a0fd-1471bc40ca17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:09.164406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d321ac3-fd35-4446-805c-1f232b55a0fe admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:09.166277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0d321ac3-fd35-4446-805c-1f232b55a0fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:09.180426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:09.212999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 941/1890] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:09.220644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d736f13-5ab9-413a-a0fd-1471bc40ca17 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:09.222063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2d736f13-5ab9-413a-a0fd-1471bc40ca17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:09.252894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d321ac3-fd35-4446-805c-1f232b55a0fe admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:09.258020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:09.259695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:09.259695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:09.259962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] PATH_INFO: `/v3/projects/808f3e9aaa2e47a1b34a9a9e48101c0e` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:09.261330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=808f3e9aaa2e47a1b34a9a9e48101c0e)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:09.263613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:09.280722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d736f13-5ab9-413a-a0fd-1471bc40ca17 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:09.284470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6068239-ca8c-41a1-9bb3-27f1aa490de2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:09.285620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6068239-ca8c-41a1-9bb3-27f1aa490de2 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:09.285903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6068239-ca8c-41a1-9bb3-27f1aa490de2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:09.286179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6068239-ca8c-41a1-9bb3-27f1aa490de2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:09.295962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6068239-ca8c-41a1-9bb3-27f1aa490de2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:09.297393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6068239-ca8c-41a1-9bb3-27f1aa490de2 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:09.307755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 942/1891] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:09.313157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '808f3e9aaa2e47a1b34a9a9e48101c0e', 'request_id': 'req-255f6f4b-8b61-4efc-918d-a05867c22c0f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:09.322939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '808f3e9aaa2e47a1b34a9a9e48101c0e', 'request_id': 'req-255f6f4b-8b61-4efc-918d-a05867c22c0f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:09.330943 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '808f3e9aaa2e47a1b34a9a9e48101c0e', 'request_id': 'req-255f6f4b-8b61-4efc-918d-a05867c22c0f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:09.342588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] The token cache is being invalidate because project 808f3e9aaa2e47a1b34a9a9e48101c0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:09.343020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:09.343851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:09.345588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 950/1892] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:09 2022] DELETE /identity/v3/projects/808f3e9aaa2e47a1b34a9a9e48101c0e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:09.349910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6068239-ca8c-41a1-9bb3-27f1aa490de2 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:09.361816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6068239-ca8c-41a1-9bb3-27f1aa490de2 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:09.418726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:09.420241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:09.426167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:09.427250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:09.427542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:09.427820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] PATH_INFO: `/v3/users/53b3618ae3174904bdfa4dc823506ef1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:09.429033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53b3618ae3174904bdfa4dc823506ef1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:09.430960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:09.478760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53b3618ae3174904bdfa4dc823506ef1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:09.479449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53b3618ae3174904bdfa4dc823506ef1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:09.480064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53b3618ae3174904bdfa4dc823506ef1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:09.488369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f6f4b-8b61-4efc-918d-a05867c22c0f admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:09.489070 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53b3618ae3174904bdfa4dc823506ef1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:09.492230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f39e8-ef3a-47de-9548-e4c45d704566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:09.493045 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f39e8-ef3a-47de-9548-e4c45d704566 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:09.493367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f39e8-ef3a-47de-9548-e4c45d704566 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:09.493647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f39e8-ef3a-47de-9548-e4c45d704566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:09.495545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-828f39e8-ef3a-47de-9548-e4c45d704566 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:09.496954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-dd47483b-9241-4e58-9edc-718d8f1c7591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53b3618ae3174904bdfa4dc823506ef1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:09.518404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 943/1893] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:09 2022] DELETE /identity/v3/users/53b3618ae3174904bdfa4dc823506ef1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:09.532269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:09.586240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f39e8-ef3a-47de-9548-e4c45d704566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:09.588150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f39e8-ef3a-47de-9548-e4c45d704566 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:09.610696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 951/1894] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:10.200646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223eaed-c9d2-46c0-839c-4b50f0edb277 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.201079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223eaed-c9d2-46c0-839c-4b50f0edb277 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.201256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223eaed-c9d2-46c0-839c-4b50f0edb277 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.202858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 944/1895] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:14:10.211374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038c1fb-54ec-464e-8a58-53d092858f5a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.212184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038c1fb-54ec-464e-8a58-53d092858f5a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.212290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038c1fb-54ec-464e-8a58-53d092858f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.233314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-f038c1fb-54ec-464e-8a58-53d092858f5a None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:10.237384 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f038c1fb-54ec-464e-8a58-53d092858f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:10.243718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223eaed-c9d2-46c0-839c-4b50f0edb277 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:10.245965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d223eaed-c9d2-46c0-839c-4b50f0edb277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:10.269483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:10.371565 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 952/1896] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:10.380656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223eaed-c9d2-46c0-839c-4b50f0edb277 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:10.385462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:10.386333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.386588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038c1fb-54ec-464e-8a58-53d092858f5a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:10.386790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.387242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] PATH_INFO: `/v3/projects/f62fd70c29084d6b9a8a6c872c0f6236` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.388281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f038c1fb-54ec-464e-8a58-53d092858f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:10.388548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f62fd70c29084d6b9a8a6c872c0f6236)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:10.390430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:10.403242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038c1fb-54ec-464e-8a58-53d092858f5a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:10.412779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60774e67-d170-4081-bbf0-aa0ea30465b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:10.413720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60774e67-d170-4081-bbf0-aa0ea30465b6 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.414061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60774e67-d170-4081-bbf0-aa0ea30465b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.414445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60774e67-d170-4081-bbf0-aa0ea30465b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.425307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60774e67-d170-4081-bbf0-aa0ea30465b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:10.427728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60774e67-d170-4081-bbf0-aa0ea30465b6 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:10.440492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 953/1897] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:10.469522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f62fd70c29084d6b9a8a6c872c0f6236', 'request_id': 'req-fc7d5e9e-af56-496e-aa10-4cdc8b780700'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:10.479459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f62fd70c29084d6b9a8a6c872c0f6236', 'request_id': 'req-fc7d5e9e-af56-496e-aa10-4cdc8b780700'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:10.486556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f62fd70c29084d6b9a8a6c872c0f6236', 'request_id': 'req-fc7d5e9e-af56-496e-aa10-4cdc8b780700'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:10.498413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] The token cache is being invalidate because project f62fd70c29084d6b9a8a6c872c0f6236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:10.498716 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:10.499391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fc7d5e9e-af56-496e-aa10-4cdc8b780700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:10.505268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 945/1898] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:10 2022] DELETE /identity/v3/projects/f62fd70c29084d6b9a8a6c872c0f6236 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:10.506872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f82f-3f3b-4b92-82c8-9a54f195f0f4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.507233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f82f-3f3b-4b92-82c8-9a54f195f0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.507512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f82f-3f3b-4b92-82c8-9a54f195f0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.529624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-71c0f82f-3f3b-4b92-82c8-9a54f195f0f4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:10.531289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-71c0f82f-3f3b-4b92-82c8-9a54f195f0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:10.595879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 954/1899] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:10.603746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6e27c-2c11-4e07-a1c8-275a403a7ad0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.604044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6e27c-2c11-4e07-a1c8-275a403a7ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.604327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6e27c-2c11-4e07-a1c8-275a403a7ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.623065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a4c6e27c-2c11-4e07-a1c8-275a403a7ad0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:10.624284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a4c6e27c-2c11-4e07-a1c8-275a403a7ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:10.634901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 946/1900] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:10.637170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0f82f-3f3b-4b92-82c8-9a54f195f0f4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:10.646903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0f82f-3f3b-4b92-82c8-9a54f195f0f4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:10.651515 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ebdae-f950-4700-a9f6-b99fb8c45674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:10.652161 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ebdae-f950-4700-a9f6-b99fb8c45674 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.652444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ebdae-f950-4700-a9f6-b99fb8c45674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.652720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286ebdae-f950-4700-a9f6-b99fb8c45674 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.653446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ebdae-f950-4700-a9f6-b99fb8c45674 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:10.653758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ebdae-f950-4700-a9f6-b99fb8c45674 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:10.655387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286ebdae-f950-4700-a9f6-b99fb8c45674 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:10.666919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 955/1901] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:14:10.671478 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6e27c-2c11-4e07-a1c8-275a403a7ad0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:10.681258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6e27c-2c11-4e07-a1c8-275a403a7ad0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:10.686065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d92bc4-ab4f-4f38-af07-6197eb57e0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:10.686763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d92bc4-ab4f-4f38-af07-6197eb57e0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.687071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d92bc4-ab4f-4f38-af07-6197eb57e0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.687361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d92bc4-ab4f-4f38-af07-6197eb57e0b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.688058 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d92bc4-ab4f-4f38-af07-6197eb57e0b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:10.690077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d92bc4-ab4f-4f38-af07-6197eb57e0b7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:10.716732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 947/1902] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:10 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:10.720934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ebdae-f950-4700-a9f6-b99fb8c45674 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:10.732345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286ebdae-f950-4700-a9f6-b99fb8c45674 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:10.766791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:10.790258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33eda2f-830f-4398-8eec-444a901f7dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:10.790937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33eda2f-830f-4398-8eec-444a901f7dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.791249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33eda2f-830f-4398-8eec-444a901f7dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.791531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33eda2f-830f-4398-8eec-444a901f7dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.792239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33eda2f-830f-4398-8eec-444a901f7dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:10.794174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33eda2f-830f-4398-8eec-444a901f7dc1 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:10.815389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 956/1903] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:10 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:10.819983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d92bc4-ab4f-4f38-af07-6197eb57e0b7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:10.830604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d92bc4-ab4f-4f38-af07-6197eb57e0b7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:10.835532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ee5fc-d544-4a0e-a1b7-32f23aa6a8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:10.836200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ee5fc-d544-4a0e-a1b7-32f23aa6a8ab admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.836492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ee5fc-d544-4a0e-a1b7-32f23aa6a8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.837034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ee5fc-d544-4a0e-a1b7-32f23aa6a8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.837795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ee5fc-d544-4a0e-a1b7-32f23aa6a8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:10.839695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ee5fc-d544-4a0e-a1b7-32f23aa6a8ab admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:10.851855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 948/1904] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:10.855802 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33eda2f-830f-4398-8eec-444a901f7dc1 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:10.866276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33eda2f-830f-4398-8eec-444a901f7dc1 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:10.870855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbb0589-3d0d-4cb5-9a63-8f1030712d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:10.871550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb0589-3d0d-4cb5-9a63-8f1030712d4f admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.871838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb0589-3d0d-4cb5-9a63-8f1030712d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.872116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbb0589-3d0d-4cb5-9a63-8f1030712d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.872838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb0589-3d0d-4cb5-9a63-8f1030712d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:10.874422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbb0589-3d0d-4cb5-9a63-8f1030712d4f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:10.887542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 957/1905] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:10.892044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ee5fc-d544-4a0e-a1b7-32f23aa6a8ab admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:10.902750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ee5fc-d544-4a0e-a1b7-32f23aa6a8ab admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:10.907383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe86dc8-2d1a-4da3-ad04-912e7764ce24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:10.908268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe86dc8-2d1a-4da3-ad04-912e7764ce24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.908561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe86dc8-2d1a-4da3-ad04-912e7764ce24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.908840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe86dc8-2d1a-4da3-ad04-912e7764ce24 admin admin] PATH_INFO: `/v3/projects/a405d9a1cb1b497f8a4187245cf5cab4/users/06ac081b00184db99f6c0cc0b6a7df35/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.923993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe86dc8-2d1a-4da3-ad04-912e7764ce24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a405d9a1cb1b497f8a4187245cf5cab4, user_id=06ac081b00184db99f6c0cc0b6a7df35, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:10.927371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe86dc8-2d1a-4da3-ad04-912e7764ce24 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:10.941142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 949/1906] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:10 2022] PUT /identity/v3/projects/a405d9a1cb1b497f8a4187245cf5cab4/users/06ac081b00184db99f6c0cc0b6a7df35/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:10.952569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50c857-0c84-4e26-82c7-56e24b538a6f None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:10.952862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50c857-0c84-4e26-82c7-56e24b538a6f None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:10.953144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50c857-0c84-4e26-82c7-56e24b538a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:10.980814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-7b50c857-0c84-4e26-82c7-56e24b538a6f None None] MFA Rules not processed for user `06ac081b00184db99f6c0cc0b6a7df35`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:10.982033 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7b50c857-0c84-4e26-82c7-56e24b538a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:11.063444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 958/1907] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:11.068055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe86dc8-2d1a-4da3-ad04-912e7764ce24 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:11.164481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:11.220781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe86dc8-2d1a-4da3-ad04-912e7764ce24 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:11.239343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321df5e0-c961-4d9c-b407-0870beb09ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:11.240046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321df5e0-c961-4d9c-b407-0870beb09ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:11.240341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321df5e0-c961-4d9c-b407-0870beb09ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:11.240623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321df5e0-c961-4d9c-b407-0870beb09ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:11.241349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321df5e0-c961-4d9c-b407-0870beb09ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:11.243322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321df5e0-c961-4d9c-b407-0870beb09ca6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:11.315522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 950/1908] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:11 2022] POST /identity/v3/projects => generated 377 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:11.317010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50c857-0c84-4e26-82c7-56e24b538a6f None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:11.335581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50c857-0c84-4e26-82c7-56e24b538a6f None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:11.429377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81f043-b52f-4809-996f-96268ac78ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:11.430063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81f043-b52f-4809-996f-96268ac78ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:11.430354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81f043-b52f-4809-996f-96268ac78ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:11.430637 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81f043-b52f-4809-996f-96268ac78ba0 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:11.431460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81f043-b52f-4809-996f-96268ac78ba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:11.433563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab81f043-b52f-4809-996f-96268ac78ba0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:11.467656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 959/1909] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:11 2022] POST /identity/v3/users => generated 289 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:11.474409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321df5e0-c961-4d9c-b407-0870beb09ca6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:11.485453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321df5e0-c961-4d9c-b407-0870beb09ca6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:11.492298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a11b4-1c72-4c85-8aa3-e94ce58c4d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:11.494243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a11b4-1c72-4c85-8aa3-e94ce58c4d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:11.494243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a11b4-1c72-4c85-8aa3-e94ce58c4d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:11.494243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82a11b4-1c72-4c85-8aa3-e94ce58c4d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:11.494243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a11b4-1c72-4c85-8aa3-e94ce58c4d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:11.495916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82a11b4-1c72-4c85-8aa3-e94ce58c4d39 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:11.510437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 951/1910] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:11.514887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81f043-b52f-4809-996f-96268ac78ba0 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:11.531576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab81f043-b52f-4809-996f-96268ac78ba0 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:11.541318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14478855-a846-4af9-a633-008ab5e8314d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:11.541995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14478855-a846-4af9-a633-008ab5e8314d admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:11.542614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14478855-a846-4af9-a633-008ab5e8314d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:11.542905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14478855-a846-4af9-a633-008ab5e8314d admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:11.543835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14478855-a846-4af9-a633-008ab5e8314d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:11.549958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14478855-a846-4af9-a633-008ab5e8314d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:11.570796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 960/1911] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:14:11.573977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a11b4-1c72-4c85-8aa3-e94ce58c4d39 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:11.589631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82a11b4-1c72-4c85-8aa3-e94ce58c4d39 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:11.612545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7753e9f-5dc3-4686-a4bd-3e9499df029f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:11.615148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753e9f-5dc3-4686-a4bd-3e9499df029f admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:11.615148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753e9f-5dc3-4686-a4bd-3e9499df029f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:11.615148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7753e9f-5dc3-4686-a4bd-3e9499df029f admin admin] PATH_INFO: `/v3/projects/b2ab964c6bb246aebfa9e5f1ab03ec19/users/016920b63b884497806556a3d7f19f4a/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:11.659075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7753e9f-5dc3-4686-a4bd-3e9499df029f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ab964c6bb246aebfa9e5f1ab03ec19, user_id=016920b63b884497806556a3d7f19f4a, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:11.667672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7753e9f-5dc3-4686-a4bd-3e9499df029f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:11.709359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 952/1912] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:11 2022] PUT /identity/v3/projects/b2ab964c6bb246aebfa9e5f1ab03ec19/users/016920b63b884497806556a3d7f19f4a/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:11.711100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b57b1-f46e-4179-ba72-9d0282ba789b None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:11.711401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b57b1-f46e-4179-ba72-9d0282ba789b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:11.711936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b57b1-f46e-4179-ba72-9d0282ba789b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:11.770875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c53b57b1-f46e-4179-ba72-9d0282ba789b None None] MFA Rules not processed for user `016920b63b884497806556a3d7f19f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:11.776365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c53b57b1-f46e-4179-ba72-9d0282ba789b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:11.823606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:11.962324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 961/1913] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:11.963420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7753e9f-5dc3-4686-a4bd-3e9499df029f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:12.054408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7753e9f-5dc3-4686-a4bd-3e9499df029f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:12.059593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a9b7-55ca-49c0-ad1f-23a55a58f95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:12.060324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a9b7-55ca-49c0-ad1f-23a55a58f95d admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.060635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a9b7-55ca-49c0-ad1f-23a55a58f95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.060914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4a9b7-55ca-49c0-ad1f-23a55a58f95d admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.061621 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4a9b7-55ca-49c0-ad1f-23a55a58f95d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:12.063613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4a9b7-55ca-49c0-ad1f-23a55a58f95d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:12.091034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 953/1914] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:11 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:12.099087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b57b1-f46e-4179-ba72-9d0282ba789b None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:12.110388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b57b1-f46e-4179-ba72-9d0282ba789b None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:12.170169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eba1b9-e686-4108-836f-d447ab8a4dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:12.170934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eba1b9-e686-4108-836f-d447ab8a4dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.171331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eba1b9-e686-4108-836f-d447ab8a4dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.171704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eba1b9-e686-4108-836f-d447ab8a4dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.172478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eba1b9-e686-4108-836f-d447ab8a4dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:12.174720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eba1b9-e686-4108-836f-d447ab8a4dc8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:12.197536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 962/1915] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:12 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:12.203209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a9b7-55ca-49c0-ad1f-23a55a58f95d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:12.212559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4a9b7-55ca-49c0-ad1f-23a55a58f95d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:12.217164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2663ed-28c5-452f-9d5c-679a8ba874ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:12.217830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2663ed-28c5-452f-9d5c-679a8ba874ab admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.218620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2663ed-28c5-452f-9d5c-679a8ba874ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.218919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2663ed-28c5-452f-9d5c-679a8ba874ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.219702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2663ed-28c5-452f-9d5c-679a8ba874ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:12.221305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2663ed-28c5-452f-9d5c-679a8ba874ab admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:12.232466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 954/1916] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:14:12.236847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eba1b9-e686-4108-836f-d447ab8a4dc8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:12.247130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eba1b9-e686-4108-836f-d447ab8a4dc8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:12.252421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ba496-3921-4bce-a11c-7d38fba26f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:12.253322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ba496-3921-4bce-a11c-7d38fba26f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.254096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ba496-3921-4bce-a11c-7d38fba26f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.254374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ba496-3921-4bce-a11c-7d38fba26f2b admin admin] PATH_INFO: `/v3/projects/50cea8ac2f154c0aab73f64eb25a701c/users/b0ecebb9635940dabf9a6e67fb06edc0/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.269932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ba496-3921-4bce-a11c-7d38fba26f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cea8ac2f154c0aab73f64eb25a701c, user_id=b0ecebb9635940dabf9a6e67fb06edc0, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:12.272614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ba496-3921-4bce-a11c-7d38fba26f2b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:12.285680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 963/1917] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:12 2022] PUT /identity/v3/projects/50cea8ac2f154c0aab73f64eb25a701c/users/b0ecebb9635940dabf9a6e67fb06edc0/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:12.289612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2663ed-28c5-452f-9d5c-679a8ba874ab admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:12.299765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2663ed-28c5-452f-9d5c-679a8ba874ab admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:12.358374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afc652-d62f-4bcb-91b5-83c8285eda4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:12.359073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc652-d62f-4bcb-91b5-83c8285eda4b admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.359369 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc652-d62f-4bcb-91b5-83c8285eda4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.359669 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afc652-d62f-4bcb-91b5-83c8285eda4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.360382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc652-d62f-4bcb-91b5-83c8285eda4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:12.362177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25afc652-d62f-4bcb-91b5-83c8285eda4b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:12.373011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 955/1918] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:12.377445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ba496-3921-4bce-a11c-7d38fba26f2b admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:12.387726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ba496-3921-4bce-a11c-7d38fba26f2b admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:12.392265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76803087-186a-40f6-912a-b4def10970ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:12.393136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803087-186a-40f6-912a-b4def10970ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.393419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803087-186a-40f6-912a-b4def10970ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.393694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76803087-186a-40f6-912a-b4def10970ee admin admin] PATH_INFO: `/v3/projects/50cea8ac2f154c0aab73f64eb25a701c/users/b0ecebb9635940dabf9a6e67fb06edc0/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.395663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76803087-186a-40f6-912a-b4def10970ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=50cea8ac2f154c0aab73f64eb25a701c, user_id=b0ecebb9635940dabf9a6e67fb06edc0, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:12.398278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76803087-186a-40f6-912a-b4def10970ee admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:12.408829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 964/1919] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:12 2022] PUT /identity/v3/projects/50cea8ac2f154c0aab73f64eb25a701c/users/b0ecebb9635940dabf9a6e67fb06edc0/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:12.420838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f23d-ab56-4dbe-88ce-63b83108125d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.421139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f23d-ab56-4dbe-88ce-63b83108125d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.421429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb3f23d-ab56-4dbe-88ce-63b83108125d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.428218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:12.435307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76803087-186a-40f6-912a-b4def10970ee admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:12.437519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-76803087-186a-40f6-912a-b4def10970ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:12.452241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-6bb3f23d-ab56-4dbe-88ce-63b83108125d None None] MFA Rules not processed for user `b0ecebb9635940dabf9a6e67fb06edc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:12.453441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6bb3f23d-ab56-4dbe-88ce-63b83108125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:12.516214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76803087-186a-40f6-912a-b4def10970ee admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:12.521777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:12.522672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.522961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.523278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] PATH_INFO: `/v3/users/2c699a701d1243b98d84ef6f95ea619f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.524494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c699a701d1243b98d84ef6f95ea619f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:12.526444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:12.536602 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:12.563924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 956/1920] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:12.576435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3f23d-ab56-4dbe-88ce-63b83108125d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:12.577837 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6bb3f23d-ab56-4dbe-88ce-63b83108125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:12.593365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c699a701d1243b98d84ef6f95ea619f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:12.594133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c699a701d1243b98d84ef6f95ea619f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:12.595130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c699a701d1243b98d84ef6f95ea619f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:12.605364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c699a701d1243b98d84ef6f95ea619f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:12.615640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-45dbe7ca-b9f2-4ac6-902a-f5179d675472 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c699a701d1243b98d84ef6f95ea619f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:12.649929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 965/1921] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:12 2022] DELETE /identity/v3/users/2c699a701d1243b98d84ef6f95ea619f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:12.651716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3a726-a959-4ad5-b02f-aab3fa5758a3 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.652007 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3a726-a959-4ad5-b02f-aab3fa5758a3 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.652291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3a726-a959-4ad5-b02f-aab3fa5758a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.652701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb3f23d-ab56-4dbe-88ce-63b83108125d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:12.675149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-3db3a726-a959-4ad5-b02f-aab3fa5758a3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:12.677342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3db3a726-a959-4ad5-b02f-aab3fa5758a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:12.723005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be0cd8-2ae7-4b3c-badc-0c5328ef649c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:12.724021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be0cd8-2ae7-4b3c-badc-0c5328ef649c service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.724528 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be0cd8-2ae7-4b3c-badc-0c5328ef649c service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.724989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be0cd8-2ae7-4b3c-badc-0c5328ef649c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.727178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a3be0cd8-2ae7-4b3c-badc-0c5328ef649c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:12.760623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 966/1922] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:12.773494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3a726-a959-4ad5-b02f-aab3fa5758a3 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:12.776889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3db3a726-a959-4ad5-b02f-aab3fa5758a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:12.808196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:12.817500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be0cd8-2ae7-4b3c-badc-0c5328ef649c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:12.818958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be0cd8-2ae7-4b3c-badc-0c5328ef649c service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:12.832570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 957/1923] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:12.851497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3a726-a959-4ad5-b02f-aab3fa5758a3 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:12.855211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b211f-4977-4887-9de0-acfe7887fb6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:12.855869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b211f-4977-4887-9de0-acfe7887fb6c service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.856157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b211f-4977-4887-9de0-acfe7887fb6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.856434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b211f-4977-4887-9de0-acfe7887fb6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.867395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b211f-4977-4887-9de0-acfe7887fb6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:12.868911 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b211f-4977-4887-9de0-acfe7887fb6c service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:12.878712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 967/1924] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:12.879269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be0cd8-2ae7-4b3c-badc-0c5328ef649c service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:12.880694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a3be0cd8-2ae7-4b3c-badc-0c5328ef649c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:12.891170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be0cd8-2ae7-4b3c-badc-0c5328ef649c service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:12.895382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd365b-5ba1-4c69-b366-476ef082bcfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:12.896035 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd365b-5ba1-4c69-b366-476ef082bcfa service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:12.896321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd365b-5ba1-4c69-b366-476ef082bcfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:12.896604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd365b-5ba1-4c69-b366-476ef082bcfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:12.898735 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-94fd365b-5ba1-4c69-b366-476ef082bcfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:12.971755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd365b-5ba1-4c69-b366-476ef082bcfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:12.974027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd365b-5ba1-4c69-b366-476ef082bcfa service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:12.984973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 958/1925] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:13.196165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b211f-4977-4887-9de0-acfe7887fb6c service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:13.198633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-954b211f-4977-4887-9de0-acfe7887fb6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:13.219887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b211f-4977-4887-9de0-acfe7887fb6c service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:13.229627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:13.230926 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:13.231260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:13.233898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] PATH_INFO: `/v3/projects/a18639598aa0484aaa6ec6d4470ec2c6` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:13.237339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a18639598aa0484aaa6ec6d4470ec2c6)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:13.242599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:13.282866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:13.314031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a18639598aa0484aaa6ec6d4470ec2c6', 'request_id': 'req-e695ebc3-7870-48ce-81cf-fddbaa12029e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:13.323923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a18639598aa0484aaa6ec6d4470ec2c6', 'request_id': 'req-e695ebc3-7870-48ce-81cf-fddbaa12029e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:13.337488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a18639598aa0484aaa6ec6d4470ec2c6', 'request_id': 'req-e695ebc3-7870-48ce-81cf-fddbaa12029e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:13.349170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] The token cache is being invalidate because project a18639598aa0484aaa6ec6d4470ec2c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:13.349541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:13.350412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:13.352182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 968/1926] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:13 2022] DELETE /identity/v3/projects/a18639598aa0484aaa6ec6d4470ec2c6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:13.359071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd365b-5ba1-4c69-b366-476ef082bcfa service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:13.374555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd365b-5ba1-4c69-b366-476ef082bcfa service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:13.440914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:13.441857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:13.442146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:13.442551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] PATH_INFO: `/v3/users/b0de31046c1c4b72a3f32e3a36444e6f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:13.444049 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0de31046c1c4b72a3f32e3a36444e6f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:13.446119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:13.497176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0de31046c1c4b72a3f32e3a36444e6f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:13.497948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0de31046c1c4b72a3f32e3a36444e6f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:13.498829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0de31046c1c4b72a3f32e3a36444e6f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:13.508159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0de31046c1c4b72a3f32e3a36444e6f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:13.515838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-392d2a38-cd52-449f-b02c-edf1cd72652e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0de31046c1c4b72a3f32e3a36444e6f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:13.535086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 959/1927] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:13 2022] DELETE /identity/v3/users/b0de31046c1c4b72a3f32e3a36444e6f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:13.870334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:13.873821 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:13.900466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e695ebc3-7870-48ce-81cf-fddbaa12029e admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:13.973640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:13.974513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:13.974798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:13.975102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] PATH_INFO: `/v3/projects/699ed244e070471492a4d749bdad661b` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:13.976298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=699ed244e070471492a4d749bdad661b)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:13.978199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.032649 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699ed244e070471492a4d749bdad661b', 'request_id': 'req-7c601df0-5c9e-438e-91cf-e698b46e11fa'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:14.041510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699ed244e070471492a4d749bdad661b', 'request_id': 'req-7c601df0-5c9e-438e-91cf-e698b46e11fa'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:14.048183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699ed244e070471492a4d749bdad661b', 'request_id': 'req-7c601df0-5c9e-438e-91cf-e698b46e11fa'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:14.059472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] The token cache is being invalidate because project 699ed244e070471492a4d749bdad661b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:14.059765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:14.060412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7c601df0-5c9e-438e-91cf-e698b46e11fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:14.061945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 969/1928] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:13 2022] DELETE /identity/v3/projects/699ed244e070471492a4d749bdad661b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:14.066819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86124b39-8377-4185-9618-a7042521c73e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.067145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86124b39-8377-4185-9618-a7042521c73e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.067434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86124b39-8377-4185-9618-a7042521c73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.085857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-86124b39-8377-4185-9618-a7042521c73e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:14.087367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-86124b39-8377-4185-9618-a7042521c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:14.092548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:14.146132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 960/1929] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:14.153852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7a74c-e7e9-4895-8218-4abc1f7fd96a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.154151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7a74c-e7e9-4895-8218-4abc1f7fd96a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.154440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f7a74c-e7e9-4895-8218-4abc1f7fd96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.173228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-01f7a74c-e7e9-4895-8218-4abc1f7fd96a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:14.174488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-01f7a74c-e7e9-4895-8218-4abc1f7fd96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:14.182509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 970/1930] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:14.186456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86124b39-8377-4185-9618-a7042521c73e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.196421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86124b39-8377-4185-9618-a7042521c73e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.201080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50042f8-ddaf-4519-9f5a-5a2d4c2025dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.201885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50042f8-ddaf-4519-9f5a-5a2d4c2025dc admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.202176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50042f8-ddaf-4519-9f5a-5a2d4c2025dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.202458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50042f8-ddaf-4519-9f5a-5a2d4c2025dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.203333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50042f8-ddaf-4519-9f5a-5a2d4c2025dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:14.203649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50042f8-ddaf-4519-9f5a-5a2d4c2025dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.205262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50042f8-ddaf-4519-9f5a-5a2d4c2025dc admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.218115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 961/1931] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:14:14.223122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7a74c-e7e9-4895-8218-4abc1f7fd96a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.232324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f7a74c-e7e9-4895-8218-4abc1f7fd96a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.237004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79f3a8-547d-4101-b269-6239dcd107e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.237651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79f3a8-547d-4101-b269-6239dcd107e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.237971 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79f3a8-547d-4101-b269-6239dcd107e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.238250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79f3a8-547d-4101-b269-6239dcd107e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.238939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79f3a8-547d-4101-b269-6239dcd107e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.240872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79f3a8-547d-4101-b269-6239dcd107e2 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.265207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 971/1932] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:14 2022] POST /identity/v3/projects => generated 387 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:14.269853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50042f8-ddaf-4519-9f5a-5a2d4c2025dc admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.279747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50042f8-ddaf-4519-9f5a-5a2d4c2025dc admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.334374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2427f-fc63-48e7-8fef-b771069d348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.335062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2427f-fc63-48e7-8fef-b771069d348e admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.335366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2427f-fc63-48e7-8fef-b771069d348e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.335647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2427f-fc63-48e7-8fef-b771069d348e admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.336338 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2427f-fc63-48e7-8fef-b771069d348e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.338570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2427f-fc63-48e7-8fef-b771069d348e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.357796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 962/1933] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:14 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:14.361645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79f3a8-547d-4101-b269-6239dcd107e2 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.371765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79f3a8-547d-4101-b269-6239dcd107e2 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.376401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a62e-915d-447b-b6b0-3c318b41a54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.377061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a62e-915d-447b-b6b0-3c318b41a54e admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.377347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a62e-915d-447b-b6b0-3c318b41a54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.377624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a62e-915d-447b-b6b0-3c318b41a54e admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.378387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608a62e-915d-447b-b6b0-3c318b41a54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.380073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608a62e-915d-447b-b6b0-3c318b41a54e admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.390500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 972/1934] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:14.395329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2427f-fc63-48e7-8fef-b771069d348e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.412284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2427f-fc63-48e7-8fef-b771069d348e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.417277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e84b1-9a19-4abd-89b6-ec2b0a82f783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.417965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e84b1-9a19-4abd-89b6-ec2b0a82f783 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.418254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e84b1-9a19-4abd-89b6-ec2b0a82f783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.418533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e84b1-9a19-4abd-89b6-ec2b0a82f783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.419381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e84b1-9a19-4abd-89b6-ec2b0a82f783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.421093 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835e84b1-9a19-4abd-89b6-ec2b0a82f783 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.432182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 963/1935] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:14.436282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a62e-915d-447b-b6b0-3c318b41a54e admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.442206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:14.449109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a62e-915d-447b-b6b0-3c318b41a54e admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.455404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de146b61-df4f-41b2-83d1-ea5e1f2ee726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.456475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de146b61-df4f-41b2-83d1-ea5e1f2ee726 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.457339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de146b61-df4f-41b2-83d1-ea5e1f2ee726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.459105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de146b61-df4f-41b2-83d1-ea5e1f2ee726 admin admin] PATH_INFO: `/v3/projects/0c8e2eea8e8d46c2aa8d1341c04f807e/users/fc885c09fce34a16abb7a967d1a6e307/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.493365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de146b61-df4f-41b2-83d1-ea5e1f2ee726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c8e2eea8e8d46c2aa8d1341c04f807e, user_id=fc885c09fce34a16abb7a967d1a6e307, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.496201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de146b61-df4f-41b2-83d1-ea5e1f2ee726 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.520376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 973/1936] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:14 2022] PUT /identity/v3/projects/0c8e2eea8e8d46c2aa8d1341c04f807e/users/fc885c09fce34a16abb7a967d1a6e307/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:14.530310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e0ca5-0419-41fd-bf91-a8ade6477d30 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.530538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e0ca5-0419-41fd-bf91-a8ade6477d30 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.530820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e0ca5-0419-41fd-bf91-a8ade6477d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.558805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e52e0ca5-0419-41fd-bf91-a8ade6477d30 None None] MFA Rules not processed for user `fc885c09fce34a16abb7a967d1a6e307`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:14.560053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e52e0ca5-0419-41fd-bf91-a8ade6477d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:14.629723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 964/1937] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:14.634102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de146b61-df4f-41b2-83d1-ea5e1f2ee726 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.698123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de146b61-df4f-41b2-83d1-ea5e1f2ee726 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.702770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3693a-85ca-4912-a7fb-7612879354ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.703456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3693a-85ca-4912-a7fb-7612879354ba admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.703745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3693a-85ca-4912-a7fb-7612879354ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.704024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3693a-85ca-4912-a7fb-7612879354ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.704716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3693a-85ca-4912-a7fb-7612879354ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.706661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3693a-85ca-4912-a7fb-7612879354ba admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.731936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 974/1938] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:14 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:14.735955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e0ca5-0419-41fd-bf91-a8ade6477d30 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.747460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e0ca5-0419-41fd-bf91-a8ade6477d30 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.750941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:14.804795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5ce60-ac49-4ddc-89bd-8fdb189d18b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.805394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ce60-ac49-4ddc-89bd-8fdb189d18b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.805682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ce60-ac49-4ddc-89bd-8fdb189d18b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.805961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5ce60-ac49-4ddc-89bd-8fdb189d18b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.806643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5ce60-ac49-4ddc-89bd-8fdb189d18b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.808578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5ce60-ac49-4ddc-89bd-8fdb189d18b8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.828034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 965/1939] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:14 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:14.832510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3693a-85ca-4912-a7fb-7612879354ba admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.842848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3693a-85ca-4912-a7fb-7612879354ba admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.848246 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c79e6b-b026-4bdb-b6fb-08ed792da949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.848910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c79e6b-b026-4bdb-b6fb-08ed792da949 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.849199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c79e6b-b026-4bdb-b6fb-08ed792da949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.849481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c79e6b-b026-4bdb-b6fb-08ed792da949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.850209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c79e6b-b026-4bdb-b6fb-08ed792da949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.851831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c79e6b-b026-4bdb-b6fb-08ed792da949 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.862447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 975/1940] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:14.866745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5ce60-ac49-4ddc-89bd-8fdb189d18b8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.876682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5ce60-ac49-4ddc-89bd-8fdb189d18b8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.881269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf16c-f16a-4e91-a2d0-6f39dfeb121f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.882148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf16c-f16a-4e91-a2d0-6f39dfeb121f admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.882432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf16c-f16a-4e91-a2d0-6f39dfeb121f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.882722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdf16c-f16a-4e91-a2d0-6f39dfeb121f admin admin] PATH_INFO: `/v3/projects/02551088867c4779b521f993e1e45c91/users/f202ae2a574b491fbcb93599ae6e7f87/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.897810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf16c-f16a-4e91-a2d0-6f39dfeb121f admin admin] RBAC: Authorizing `identity:create_grant(project_id=02551088867c4779b521f993e1e45c91, user_id=f202ae2a574b491fbcb93599ae6e7f87, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.900551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbdf16c-f16a-4e91-a2d0-6f39dfeb121f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:14.913550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 966/1941] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:14 2022] PUT /identity/v3/projects/02551088867c4779b521f993e1e45c91/users/f202ae2a574b491fbcb93599ae6e7f87/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:14.917505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c79e6b-b026-4bdb-b6fb-08ed792da949 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:14.927875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c79e6b-b026-4bdb-b6fb-08ed792da949 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:14.984495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c19d01f-ca2c-44a3-940c-9b6ca9aaa829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:14.985179 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19d01f-ca2c-44a3-940c-9b6ca9aaa829 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:14.985470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19d01f-ca2c-44a3-940c-9b6ca9aaa829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:14.985753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c19d01f-ca2c-44a3-940c-9b6ca9aaa829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:14.986490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19d01f-ca2c-44a3-940c-9b6ca9aaa829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:14.988638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c19d01f-ca2c-44a3-940c-9b6ca9aaa829 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:15.000307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 976/1942] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:14:15.005300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf16c-f16a-4e91-a2d0-6f39dfeb121f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:15.020047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdf16c-f16a-4e91-a2d0-6f39dfeb121f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:15.028015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cf446-756f-4673-afa6-c4dfb70dc727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:15.028909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cf446-756f-4673-afa6-c4dfb70dc727 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:15.029240 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cf446-756f-4673-afa6-c4dfb70dc727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:15.029455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4cf446-756f-4673-afa6-c4dfb70dc727 admin admin] PATH_INFO: `/v3/projects/02551088867c4779b521f993e1e45c91/users/f202ae2a574b491fbcb93599ae6e7f87/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:15.031519 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4cf446-756f-4673-afa6-c4dfb70dc727 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02551088867c4779b521f993e1e45c91, user_id=f202ae2a574b491fbcb93599ae6e7f87, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:15.035476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4cf446-756f-4673-afa6-c4dfb70dc727 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:15.045661 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 967/1943] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:15 2022] PUT /identity/v3/projects/02551088867c4779b521f993e1e45c91/users/f202ae2a574b491fbcb93599ae6e7f87/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:15.057198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c21aa-1286-465e-96a0-01a29596bf47 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:15.057588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c21aa-1286-465e-96a0-01a29596bf47 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:15.057969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c21aa-1286-465e-96a0-01a29596bf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:15.090042 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4a7c21aa-1286-465e-96a0-01a29596bf47 None None] MFA Rules not processed for user `f202ae2a574b491fbcb93599ae6e7f87`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:15.091294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4a7c21aa-1286-465e-96a0-01a29596bf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:15.135335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:15.168341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 977/1944] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:15.179191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cf446-756f-4673-afa6-c4dfb70dc727 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:15.181030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bd4cf446-756f-4673-afa6-c4dfb70dc727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:15.245550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4cf446-756f-4673-afa6-c4dfb70dc727 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:15.249164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b7588-30d2-47cb-865b-5f16ec3b9017 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:15.249810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b7588-30d2-47cb-865b-5f16ec3b9017 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:15.250096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b7588-30d2-47cb-865b-5f16ec3b9017 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:15.250377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b7588-30d2-47cb-865b-5f16ec3b9017 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:15.259685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b7588-30d2-47cb-865b-5f16ec3b9017 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:15.261108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b7588-30d2-47cb-865b-5f16ec3b9017 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:15.271027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 968/1945] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:15.855325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c21aa-1286-465e-96a0-01a29596bf47 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:15.917624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c21aa-1286-465e-96a0-01a29596bf47 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:15.922145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:15.923069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:15.923366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:15.923672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] PATH_INFO: `/v3/users/fc885c09fce34a16abb7a967d1a6e307` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:15.924862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc885c09fce34a16abb7a967d1a6e307)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:15.927349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:15.974762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc885c09fce34a16abb7a967d1a6e307'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:15.975441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc885c09fce34a16abb7a967d1a6e307'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:15.976073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc885c09fce34a16abb7a967d1a6e307'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:15.982854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc885c09fce34a16abb7a967d1a6e307'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:15.991655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc885c09fce34a16abb7a967d1a6e307'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.012307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 978/1946] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:15 2022] DELETE /identity/v3/users/fc885c09fce34a16abb7a967d1a6e307 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:16.016890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dc8cb-2155-4a31-9a4d-2a880cb8452c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:16.017364 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dc8cb-2155-4a31-9a4d-2a880cb8452c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:16.017479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919dc8cb-2155-4a31-9a4d-2a880cb8452c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:16.034470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-919dc8cb-2155-4a31-9a4d-2a880cb8452c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:16.036202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-919dc8cb-2155-4a31-9a4d-2a880cb8452c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:16.097717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 969/1947] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:16.107423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:16.108938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:16.171176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fbcf0d-26cb-4389-8224-01ce8815d1a9 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:16.175165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff7604-a942-447b-bb58-b14aefb3735b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:16.175938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff7604-a942-447b-bb58-b14aefb3735b service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:16.176275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff7604-a942-447b-bb58-b14aefb3735b service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:16.176623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff7604-a942-447b-bb58-b14aefb3735b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:16.187129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:16.190505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff7604-a942-447b-bb58-b14aefb3735b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:16.192932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff7604-a942-447b-bb58-b14aefb3735b service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:16.211978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 979/1948] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:16.499107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dc8cb-2155-4a31-9a4d-2a880cb8452c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:16.500633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-919dc8cb-2155-4a31-9a4d-2a880cb8452c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:16.510315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:16.516050 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919dc8cb-2155-4a31-9a4d-2a880cb8452c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:16.522671 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:16.523570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:16.523857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:16.524172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] PATH_INFO: `/v3/projects/0c8e2eea8e8d46c2aa8d1341c04f807e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:16.525337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c8e2eea8e8d46c2aa8d1341c04f807e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:16.527299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:16.575139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c8e2eea8e8d46c2aa8d1341c04f807e', 'request_id': 'req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.583834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c8e2eea8e8d46c2aa8d1341c04f807e', 'request_id': 'req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.590760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c8e2eea8e8d46c2aa8d1341c04f807e', 'request_id': 'req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.602064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] The token cache is being invalidate because project 0c8e2eea8e8d46c2aa8d1341c04f807e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:16.602351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.603024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.604581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 970/1949] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:16 2022] DELETE /identity/v3/projects/0c8e2eea8e8d46c2aa8d1341c04f807e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:16.608631 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff7604-a942-447b-bb58-b14aefb3735b service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:16.618253 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff7604-a942-447b-bb58-b14aefb3735b service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:16.672897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:16.673767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:16.674049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:16.674329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] PATH_INFO: `/v3/users/f202ae2a574b491fbcb93599ae6e7f87` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:16.676073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f202ae2a574b491fbcb93599ae6e7f87)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:16.677978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:16.723264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f202ae2a574b491fbcb93599ae6e7f87'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.723929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f202ae2a574b491fbcb93599ae6e7f87'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.724630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f202ae2a574b491fbcb93599ae6e7f87'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.731384 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f202ae2a574b491fbcb93599ae6e7f87'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.739277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-74270830-ffa1-49e6-8266-b98fdf7c7a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f202ae2a574b491fbcb93599ae6e7f87'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:16.756159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 980/1950] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:16 2022] DELETE /identity/v3/users/f202ae2a574b491fbcb93599ae6e7f87 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:17.100033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:17.102203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:17.115714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b50b3e-4b53-4d50-abcb-893d83fdfb45 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:17.178230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:17.179121 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.179406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.179684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] PATH_INFO: `/v3/projects/02551088867c4779b521f993e1e45c91` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.180860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02551088867c4779b521f993e1e45c91)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:17.182797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:17.232772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02551088867c4779b521f993e1e45c91', 'request_id': 'req-fe10ddf0-486b-4a4b-815c-e8c7b510d381'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:17.240186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02551088867c4779b521f993e1e45c91', 'request_id': 'req-fe10ddf0-486b-4a4b-815c-e8c7b510d381'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:17.246468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02551088867c4779b521f993e1e45c91', 'request_id': 'req-fe10ddf0-486b-4a4b-815c-e8c7b510d381'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:17.257196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] The token cache is being invalidate because project 02551088867c4779b521f993e1e45c91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:17.257487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:17.258114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-fe10ddf0-486b-4a4b-815c-e8c7b510d381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:17.259776 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 971/1951] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:17 2022] DELETE /identity/v3/projects/02551088867c4779b521f993e1e45c91 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:17.264759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783c534-f7bf-4f5a-bdab-9e9bff36ef9b None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.265247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783c534-f7bf-4f5a-bdab-9e9bff36ef9b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.265369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783c534-f7bf-4f5a-bdab-9e9bff36ef9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.282148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-f783c534-f7bf-4f5a-bdab-9e9bff36ef9b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:17.283374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f783c534-f7bf-4f5a-bdab-9e9bff36ef9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:17.340902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 981/1952] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:17.348606 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f80f-1782-40de-8635-e0489d46647d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.348838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f80f-1782-40de-8635-e0489d46647d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.349120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339f80f-1782-40de-8635-e0489d46647d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.367610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c339f80f-1782-40de-8635-e0489d46647d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:17.368817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c339f80f-1782-40de-8635-e0489d46647d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:17.376802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 972/1953] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:17.381685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783c534-f7bf-4f5a-bdab-9e9bff36ef9b None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:17.392111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783c534-f7bf-4f5a-bdab-9e9bff36ef9b None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:17.396748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c60a65-8a3b-487f-b978-fca11a64ad50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:17.397421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c60a65-8a3b-487f-b978-fca11a64ad50 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.397709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c60a65-8a3b-487f-b978-fca11a64ad50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.397987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c60a65-8a3b-487f-b978-fca11a64ad50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.398726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c60a65-8a3b-487f-b978-fca11a64ad50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:17.399067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c60a65-8a3b-487f-b978-fca11a64ad50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:17.400667 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c60a65-8a3b-487f-b978-fca11a64ad50 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:17.413139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 982/1954] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:14:17.417369 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339f80f-1782-40de-8635-e0489d46647d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:17.427463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339f80f-1782-40de-8635-e0489d46647d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:17.432309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20476d9a-883d-4322-92fc-5e7f00e0eecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:17.433055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20476d9a-883d-4322-92fc-5e7f00e0eecb admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.433456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20476d9a-883d-4322-92fc-5e7f00e0eecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.433801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20476d9a-883d-4322-92fc-5e7f00e0eecb admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.434592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20476d9a-883d-4322-92fc-5e7f00e0eecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:17.436613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20476d9a-883d-4322-92fc-5e7f00e0eecb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:17.442456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:17.470940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 973/1955] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:17 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:17.473015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c60a65-8a3b-487f-b978-fca11a64ad50 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:17.478274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:17.485310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c60a65-8a3b-487f-b978-fca11a64ad50 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:17.546602 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32984df-2fab-463b-9350-6d9cea4fcf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:17.547319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32984df-2fab-463b-9350-6d9cea4fcf9f admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.547604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32984df-2fab-463b-9350-6d9cea4fcf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.547880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32984df-2fab-463b-9350-6d9cea4fcf9f admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.548577 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32984df-2fab-463b-9350-6d9cea4fcf9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:17.551997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32984df-2fab-463b-9350-6d9cea4fcf9f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:17.575524 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 983/1956] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:17 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:17.579876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20476d9a-883d-4322-92fc-5e7f00e0eecb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:17.591151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20476d9a-883d-4322-92fc-5e7f00e0eecb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:17.595738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb5fc4-0df3-4e8e-85e6-e5ea1b94d028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:17.596398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb5fc4-0df3-4e8e-85e6-e5ea1b94d028 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.596681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb5fc4-0df3-4e8e-85e6-e5ea1b94d028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.596959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abb5fc4-0df3-4e8e-85e6-e5ea1b94d028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.598232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb5fc4-0df3-4e8e-85e6-e5ea1b94d028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:17.599872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abb5fc4-0df3-4e8e-85e6-e5ea1b94d028 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:17.611086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 974/1957] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:14:17.614996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32984df-2fab-463b-9350-6d9cea4fcf9f admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:17.629313 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32984df-2fab-463b-9350-6d9cea4fcf9f admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:17.633998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a006ff9a-5b9b-4cd1-a19c-b1a0af52a3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:17.634670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006ff9a-5b9b-4cd1-a19c-b1a0af52a3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.635031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006ff9a-5b9b-4cd1-a19c-b1a0af52a3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.635250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a006ff9a-5b9b-4cd1-a19c-b1a0af52a3f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.635980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006ff9a-5b9b-4cd1-a19c-b1a0af52a3f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:17.637568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a006ff9a-5b9b-4cd1-a19c-b1a0af52a3f5 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:17.651130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 984/1958] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:17.652260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb5fc4-0df3-4e8e-85e6-e5ea1b94d028 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:17.662880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abb5fc4-0df3-4e8e-85e6-e5ea1b94d028 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:17.667540 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2d210-abf0-42f5-af73-419b4fc9485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:17.668424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2d210-abf0-42f5-af73-419b4fc9485c admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.668712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2d210-abf0-42f5-af73-419b4fc9485c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.669023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2d210-abf0-42f5-af73-419b4fc9485c admin admin] PATH_INFO: `/v3/projects/8221b8e33b3744149f117015b7554365/users/1870093b884244d692fc485c70f90568/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.683827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2d210-abf0-42f5-af73-419b4fc9485c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8221b8e33b3744149f117015b7554365, user_id=1870093b884244d692fc485c70f90568, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:17.686493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2d210-abf0-42f5-af73-419b4fc9485c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:17.699167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 975/1959] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:17 2022] PUT /identity/v3/projects/8221b8e33b3744149f117015b7554365/users/1870093b884244d692fc485c70f90568/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:17.720823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dfcb7-c7a3-4bee-bc7d-2d821bb098af None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.720823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dfcb7-c7a3-4bee-bc7d-2d821bb098af None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.720823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dfcb7-c7a3-4bee-bc7d-2d821bb098af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.747255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-240dfcb7-c7a3-4bee-bc7d-2d821bb098af None None] MFA Rules not processed for user `1870093b884244d692fc485c70f90568`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:17.748467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-240dfcb7-c7a3-4bee-bc7d-2d821bb098af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:17.812083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 985/1960] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:17.816634 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2d210-abf0-42f5-af73-419b4fc9485c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:17.881247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2d210-abf0-42f5-af73-419b4fc9485c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:17.886528 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6982227-bd27-43a7-8fc0-097b900a8db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:17.887244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982227-bd27-43a7-8fc0-097b900a8db8 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.887529 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982227-bd27-43a7-8fc0-097b900a8db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.887807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982227-bd27-43a7-8fc0-097b900a8db8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.888520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6982227-bd27-43a7-8fc0-097b900a8db8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:17.890423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6982227-bd27-43a7-8fc0-097b900a8db8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:17.918542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 976/1961] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:17 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:17.922563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dfcb7-c7a3-4bee-bc7d-2d821bb098af None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:17.933560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dfcb7-c7a3-4bee-bc7d-2d821bb098af None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:17.992611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e3235-8fa0-4480-9cfd-fe5a132d9ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:17.993613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3235-8fa0-4480-9cfd-fe5a132d9ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:17.994106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3235-8fa0-4480-9cfd-fe5a132d9ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:17.994599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e3235-8fa0-4480-9cfd-fe5a132d9ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:17.995367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e3235-8fa0-4480-9cfd-fe5a132d9ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:17.997706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3e3235-8fa0-4480-9cfd-fe5a132d9ca6 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:18.014461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:18.024341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 986/1962] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:17 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:18.032873 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6982227-bd27-43a7-8fc0-097b900a8db8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:18.039299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6982227-bd27-43a7-8fc0-097b900a8db8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:18.043836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e59ad9-c3c5-46fc-8c28-d4b41ae51b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:18.044495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59ad9-c3c5-46fc-8c28-d4b41ae51b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:18.044778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59ad9-c3c5-46fc-8c28-d4b41ae51b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:18.045053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59ad9-c3c5-46fc-8c28-d4b41ae51b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:18.045778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e59ad9-c3c5-46fc-8c28-d4b41ae51b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:18.047563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e59ad9-c3c5-46fc-8c28-d4b41ae51b86 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:18.059217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 977/1963] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:18.063412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e3235-8fa0-4480-9cfd-fe5a132d9ca6 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:18.073310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e3235-8fa0-4480-9cfd-fe5a132d9ca6 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:18.078065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c903e-6af8-40ad-94b8-db48e3810807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:18.079070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c903e-6af8-40ad-94b8-db48e3810807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:18.079362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c903e-6af8-40ad-94b8-db48e3810807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:18.079668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c903e-6af8-40ad-94b8-db48e3810807 admin admin] PATH_INFO: `/v3/projects/71aad8f3018e4671adcde2c595ae6e90/users/35cec6b09a7e40f98de7857734216def/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:18.098604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c903e-6af8-40ad-94b8-db48e3810807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71aad8f3018e4671adcde2c595ae6e90, user_id=35cec6b09a7e40f98de7857734216def, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:18.102866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c903e-6af8-40ad-94b8-db48e3810807 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:18.117159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 987/1964] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:18 2022] PUT /identity/v3/projects/71aad8f3018e4671adcde2c595ae6e90/users/35cec6b09a7e40f98de7857734216def/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:18.121126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e59ad9-c3c5-46fc-8c28-d4b41ae51b86 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:18.134204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e59ad9-c3c5-46fc-8c28-d4b41ae51b86 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:18.212370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10da379f-35aa-4c63-9196-f734b6ab348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:18.212370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da379f-35aa-4c63-9196-f734b6ab348f admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:18.212370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da379f-35aa-4c63-9196-f734b6ab348f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:18.212370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10da379f-35aa-4c63-9196-f734b6ab348f admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:18.212824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da379f-35aa-4c63-9196-f734b6ab348f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:18.214318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10da379f-35aa-4c63-9196-f734b6ab348f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:18.230695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 978/1965] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:18.235011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c903e-6af8-40ad-94b8-db48e3810807 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:18.248154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c903e-6af8-40ad-94b8-db48e3810807 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:18.253848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5715e83-6521-48da-a2e2-f3897bbd1e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:18.254758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5715e83-6521-48da-a2e2-f3897bbd1e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:18.255075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5715e83-6521-48da-a2e2-f3897bbd1e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:18.255350 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5715e83-6521-48da-a2e2-f3897bbd1e9e admin admin] PATH_INFO: `/v3/projects/71aad8f3018e4671adcde2c595ae6e90/users/35cec6b09a7e40f98de7857734216def/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:18.257461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5715e83-6521-48da-a2e2-f3897bbd1e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71aad8f3018e4671adcde2c595ae6e90, user_id=35cec6b09a7e40f98de7857734216def, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:18.260263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5715e83-6521-48da-a2e2-f3897bbd1e9e admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:18.272176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 988/1966] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:18 2022] PUT /identity/v3/projects/71aad8f3018e4671adcde2c595ae6e90/users/35cec6b09a7e40f98de7857734216def/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:18.283225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1511b-fd1a-4c08-96e9-e5f2f95322e2 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:18.283522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1511b-fd1a-4c08-96e9-e5f2f95322e2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:18.283803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1511b-fd1a-4c08-96e9-e5f2f95322e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:18.314197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-29e1511b-fd1a-4c08-96e9-e5f2f95322e2 None None] MFA Rules not processed for user `35cec6b09a7e40f98de7857734216def`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:18.315447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-29e1511b-fd1a-4c08-96e9-e5f2f95322e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:18.326937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:18.392915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 979/1967] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:18.405118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5715e83-6521-48da-a2e2-f3897bbd1e9e admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:18.406842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f5715e83-6521-48da-a2e2-f3897bbd1e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:18.464433 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5715e83-6521-48da-a2e2-f3897bbd1e9e admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:18.468347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1d75b-142d-41d2-b1c0-c7456b6ed938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:18.468997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1d75b-142d-41d2-b1c0-c7456b6ed938 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:18.469290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1d75b-142d-41d2-b1c0-c7456b6ed938 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:18.469597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1d75b-142d-41d2-b1c0-c7456b6ed938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:18.479629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1d75b-142d-41d2-b1c0-c7456b6ed938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:18.481090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f1d75b-142d-41d2-b1c0-c7456b6ed938 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:18.490567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 989/1968] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:18.577967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1511b-fd1a-4c08-96e9-e5f2f95322e2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:18.592845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1511b-fd1a-4c08-96e9-e5f2f95322e2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:18.597935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa31a3f-4b37-42d0-8faf-852a555f2b50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:18.598656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa31a3f-4b37-42d0-8faf-852a555f2b50 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:18.599018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa31a3f-4b37-42d0-8faf-852a555f2b50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:18.599398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa31a3f-4b37-42d0-8faf-852a555f2b50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:18.666410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa31a3f-4b37-42d0-8faf-852a555f2b50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:18.667834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa31a3f-4b37-42d0-8faf-852a555f2b50 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:18.691957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 980/1969] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:19.005430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1d75b-142d-41d2-b1c0-c7456b6ed938 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:19.075713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1d75b-142d-41d2-b1c0-c7456b6ed938 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:19.083170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:19.084965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:19.084965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:19.084965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] PATH_INFO: `/v3/users/1870093b884244d692fc485c70f90568` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:19.086009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1870093b884244d692fc485c70f90568)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:19.088352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:19.137882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1870093b884244d692fc485c70f90568'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:19.138538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1870093b884244d692fc485c70f90568'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:19.139176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1870093b884244d692fc485c70f90568'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:19.146413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1870093b884244d692fc485c70f90568'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:19.154096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1870093b884244d692fc485c70f90568'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:19.158691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:19.173704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 990/1970] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:19 2022] DELETE /identity/v3/users/1870093b884244d692fc485c70f90568 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:19.178233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c540b-0815-4d33-be86-c7a943f5dd88 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:19.178715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c540b-0815-4d33-be86-c7a943f5dd88 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:19.178816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c540b-0815-4d33-be86-c7a943f5dd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:19.197091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-de4c540b-0815-4d33-be86-c7a943f5dd88 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:19.200153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-de4c540b-0815-4d33-be86-c7a943f5dd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:19.262954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 981/1971] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:14:19.273009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:19.274406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:19.328820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313221d5-4ac6-4766-aa30-3275fe271cf8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:19.333056 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132cdd8-3d5a-44ef-a9e4-2a345c4de3e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:19.333693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132cdd8-3d5a-44ef-a9e4-2a345c4de3e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:19.333978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132cdd8-3d5a-44ef-a9e4-2a345c4de3e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:19.334256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132cdd8-3d5a-44ef-a9e4-2a345c4de3e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:19.344767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132cdd8-3d5a-44ef-a9e4-2a345c4de3e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:19.346254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132cdd8-3d5a-44ef-a9e4-2a345c4de3e9 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:19.356038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 991/1972] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:19.777203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c540b-0815-4d33-be86-c7a943f5dd88 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:19.778676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-de4c540b-0815-4d33-be86-c7a943f5dd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:19.793425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c540b-0815-4d33-be86-c7a943f5dd88 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:19.798569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:19.799467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:19.799752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:19.800029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] PATH_INFO: `/v3/projects/8221b8e33b3744149f117015b7554365` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:19.801319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8221b8e33b3744149f117015b7554365)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:19.803258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:19.848708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:19.855634 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8221b8e33b3744149f117015b7554365', 'request_id': 'req-e9bc840c-4698-4f99-9702-d9a8477702d8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:19.865827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8221b8e33b3744149f117015b7554365', 'request_id': 'req-e9bc840c-4698-4f99-9702-d9a8477702d8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:19.872495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8221b8e33b3744149f117015b7554365', 'request_id': 'req-e9bc840c-4698-4f99-9702-d9a8477702d8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:19.887469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] The token cache is being invalidate because project 8221b8e33b3744149f117015b7554365 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:19.887768 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:19.888680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:19.890648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 982/1973] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:19 2022] DELETE /identity/v3/projects/8221b8e33b3744149f117015b7554365 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:19.895818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132cdd8-3d5a-44ef-a9e4-2a345c4de3e9 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:19.905768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132cdd8-3d5a-44ef-a9e4-2a345c4de3e9 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:19.963674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:19.964555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:19.964836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:19.965110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] PATH_INFO: `/v3/users/35cec6b09a7e40f98de7857734216def` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:19.966300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35cec6b09a7e40f98de7857734216def)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:19.968264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:20.018882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35cec6b09a7e40f98de7857734216def'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:20.019638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35cec6b09a7e40f98de7857734216def'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:20.020214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35cec6b09a7e40f98de7857734216def'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:20.027447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35cec6b09a7e40f98de7857734216def'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:20.035697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-73f60686-a1a0-475f-99c6-37be21ea8e9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35cec6b09a7e40f98de7857734216def'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:20.054211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 992/1974] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:19 2022] DELETE /identity/v3/users/35cec6b09a7e40f98de7857734216def => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:20.335856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:20.337611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:20.350623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc840c-4698-4f99-9702-d9a8477702d8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:20.406753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:20.407660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:20.407984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:20.408267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] PATH_INFO: `/v3/projects/71aad8f3018e4671adcde2c595ae6e90` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:20.410039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=71aad8f3018e4671adcde2c595ae6e90)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:20.412046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:20.457462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71aad8f3018e4671adcde2c595ae6e90', 'request_id': 'req-4de77e77-2002-4e32-a65e-508dfb47f4ef'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:20.465033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71aad8f3018e4671adcde2c595ae6e90', 'request_id': 'req-4de77e77-2002-4e32-a65e-508dfb47f4ef'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:20.471503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71aad8f3018e4671adcde2c595ae6e90', 'request_id': 'req-4de77e77-2002-4e32-a65e-508dfb47f4ef'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:20.482312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] The token cache is being invalidate because project 71aad8f3018e4671adcde2c595ae6e90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:20.482609 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:20.483307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4de77e77-2002-4e32-a65e-508dfb47f4ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:20.484867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 983/1975] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:20 2022] DELETE /identity/v3/projects/71aad8f3018e4671adcde2c595ae6e90 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:20.490894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9756ec7-d42c-4c41-a294-f09401a61267 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:20.491389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9756ec7-d42c-4c41-a294-f09401a61267 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:20.491589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9756ec7-d42c-4c41-a294-f09401a61267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:20.504742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:20.512099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b9756ec7-d42c-4c41-a294-f09401a61267 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:20.514099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b9756ec7-d42c-4c41-a294-f09401a61267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:20.572179 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 993/1976] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:20.579588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341155-4cd3-453b-a700-389e985dd3c8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:20.579912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341155-4cd3-453b-a700-389e985dd3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:20.580199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14341155-4cd3-453b-a700-389e985dd3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:20.598838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-14341155-4cd3-453b-a700-389e985dd3c8 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:20.600066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-14341155-4cd3-453b-a700-389e985dd3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:20.607864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 984/1977] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:20.611793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9756ec7-d42c-4c41-a294-f09401a61267 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:20.622056 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9756ec7-d42c-4c41-a294-f09401a61267 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:20.627454 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76a13-4bb6-46ec-b70b-63bb128387db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:20.627454 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76a13-4bb6-46ec-b70b-63bb128387db admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:20.627942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76a13-4bb6-46ec-b70b-63bb128387db admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:20.628227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76a13-4bb6-46ec-b70b-63bb128387db admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:20.628958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76a13-4bb6-46ec-b70b-63bb128387db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:20.629263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76a13-4bb6-46ec-b70b-63bb128387db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:20.630861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76a13-4bb6-46ec-b70b-63bb128387db admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:20.643768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 994/1978] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:14:20.647921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341155-4cd3-453b-a700-389e985dd3c8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:20.657095 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14341155-4cd3-453b-a700-389e985dd3c8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:20.663186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d248fb4-6196-4af3-a8c7-c5c02bd29c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:20.663854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d248fb4-6196-4af3-a8c7-c5c02bd29c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:20.664698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d248fb4-6196-4af3-a8c7-c5c02bd29c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:20.664698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d248fb4-6196-4af3-a8c7-c5c02bd29c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:20.665108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d248fb4-6196-4af3-a8c7-c5c02bd29c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:20.667023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d248fb4-6196-4af3-a8c7-c5c02bd29c73 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:20.697111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 985/1979] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:20 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:20.698392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76a13-4bb6-46ec-b70b-63bb128387db admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:20.708788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76a13-4bb6-46ec-b70b-63bb128387db admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:20.770094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983670-e33f-4feb-be82-8e6264549be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:20.770783 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983670-e33f-4feb-be82-8e6264549be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:20.771192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983670-e33f-4feb-be82-8e6264549be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:20.771382 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b983670-e33f-4feb-be82-8e6264549be4 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:20.772279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b983670-e33f-4feb-be82-8e6264549be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:20.774202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b983670-e33f-4feb-be82-8e6264549be4 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:20.797137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 995/1980] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:20 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:20.798600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d248fb4-6196-4af3-a8c7-c5c02bd29c73 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:20.808885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d248fb4-6196-4af3-a8c7-c5c02bd29c73 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:20.813990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487b616-1d91-492c-985f-9a3337041cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:20.814983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487b616-1d91-492c-985f-9a3337041cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:20.814983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487b616-1d91-492c-985f-9a3337041cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:20.815227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487b616-1d91-492c-985f-9a3337041cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:20.816655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487b616-1d91-492c-985f-9a3337041cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:20.818374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487b616-1d91-492c-985f-9a3337041cc4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:20.831171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 986/1981] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:14:20.835397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983670-e33f-4feb-be82-8e6264549be4 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:20.846363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b983670-e33f-4feb-be82-8e6264549be4 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:20.851296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d63a9-45a9-4eb8-a919-0f233c75734c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:20.851987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d63a9-45a9-4eb8-a919-0f233c75734c admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:20.852273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d63a9-45a9-4eb8-a919-0f233c75734c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:20.852561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d63a9-45a9-4eb8-a919-0f233c75734c admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:20.853300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d63a9-45a9-4eb8-a919-0f233c75734c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:20.854925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d63a9-45a9-4eb8-a919-0f233c75734c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:20.869629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 996/1982] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:20.870973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487b616-1d91-492c-985f-9a3337041cc4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:20.882320 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487b616-1d91-492c-985f-9a3337041cc4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:20.887221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb697a7d-eb85-4a0a-8426-d20337dba988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:20.888200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb697a7d-eb85-4a0a-8426-d20337dba988 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:20.888489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb697a7d-eb85-4a0a-8426-d20337dba988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:20.888772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb697a7d-eb85-4a0a-8426-d20337dba988 admin admin] PATH_INFO: `/v3/projects/fc44f2c4d709473e82a0aa5980b55a7b/users/d7674384095149d4a7f9bf2c48068c95/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:20.904882 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb697a7d-eb85-4a0a-8426-d20337dba988 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc44f2c4d709473e82a0aa5980b55a7b, user_id=d7674384095149d4a7f9bf2c48068c95, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:20.907625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb697a7d-eb85-4a0a-8426-d20337dba988 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:20.911035 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:20.924647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 987/1983] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:20 2022] PUT /identity/v3/projects/fc44f2c4d709473e82a0aa5980b55a7b/users/d7674384095149d4a7f9bf2c48068c95/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:20.936507 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb198ef-d0e0-4cfa-9d15-98ce7480b642 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:20.936917 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb198ef-d0e0-4cfa-9d15-98ce7480b642 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:20.937273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb198ef-d0e0-4cfa-9d15-98ce7480b642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:20.967052 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ffb198ef-d0e0-4cfa-9d15-98ce7480b642 None None] MFA Rules not processed for user `d7674384095149d4a7f9bf2c48068c95`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:20.968357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ffb198ef-d0e0-4cfa-9d15-98ce7480b642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:21.039226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 997/1984] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:21.051659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb697a7d-eb85-4a0a-8426-d20337dba988 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:21.053684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cb697a7d-eb85-4a0a-8426-d20337dba988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:21.116743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb697a7d-eb85-4a0a-8426-d20337dba988 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:21.120505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d178f8e-4ec6-4182-964d-18aaf831810c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:21.121163 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178f8e-4ec6-4182-964d-18aaf831810c service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:21.121453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178f8e-4ec6-4182-964d-18aaf831810c service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:21.121738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178f8e-4ec6-4182-964d-18aaf831810c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:21.131698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d178f8e-4ec6-4182-964d-18aaf831810c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:21.133148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d178f8e-4ec6-4182-964d-18aaf831810c service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:21.142753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 988/1985] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:21.161809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb198ef-d0e0-4cfa-9d15-98ce7480b642 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:21.166401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:21.240456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb198ef-d0e0-4cfa-9d15-98ce7480b642 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:21.247285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:21.248339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:21.248909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:21.248909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] PATH_INFO: `/v3/users/d7674384095149d4a7f9bf2c48068c95` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:21.250237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7674384095149d4a7f9bf2c48068c95)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:21.254016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:21.318872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7674384095149d4a7f9bf2c48068c95'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:21.319546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7674384095149d4a7f9bf2c48068c95'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:21.320141 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7674384095149d4a7f9bf2c48068c95'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:21.326763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7674384095149d4a7f9bf2c48068c95'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:21.344060 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7674384095149d4a7f9bf2c48068c95'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:21.361250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 998/1986] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:21 2022] DELETE /identity/v3/users/d7674384095149d4a7f9bf2c48068c95 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:21.365715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7e1f3-3615-4838-8876-e30fb595c92c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:21.366165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7e1f3-3615-4838-8876-e30fb595c92c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:21.366295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7e1f3-3615-4838-8876-e30fb595c92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:21.385007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9dc7e1f3-3615-4838-8876-e30fb595c92c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:21.386210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9dc7e1f3-3615-4838-8876-e30fb595c92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:21.445218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 989/1987] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:21.454792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:21.456294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:21.511881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d42bf7-542a-47ab-b414-1e72074b8f47 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:21.515582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5432c-7055-4b01-8511-f7d7629b4d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:21.516230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5432c-7055-4b01-8511-f7d7629b4d1c service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:21.516517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5432c-7055-4b01-8511-f7d7629b4d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:21.516832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5432c-7055-4b01-8511-f7d7629b4d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:21.526369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5432c-7055-4b01-8511-f7d7629b4d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:21.527841 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5432c-7055-4b01-8511-f7d7629b4d1c service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:21.537084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 999/1988] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:21.792257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7e1f3-3615-4838-8876-e30fb595c92c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:21.793693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9dc7e1f3-3615-4838-8876-e30fb595c92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:21.806200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7e1f3-3615-4838-8876-e30fb595c92c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:21.811160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:21.812052 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:21.812361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:21.812631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] PATH_INFO: `/v3/projects/fc44f2c4d709473e82a0aa5980b55a7b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:21.813907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc44f2c4d709473e82a0aa5980b55a7b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:21.815852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:21.864044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc44f2c4d709473e82a0aa5980b55a7b', 'request_id': 'req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:21.871273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc44f2c4d709473e82a0aa5980b55a7b', 'request_id': 'req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:21.877472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc44f2c4d709473e82a0aa5980b55a7b', 'request_id': 'req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:21.888162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] The token cache is being invalidate because project fc44f2c4d709473e82a0aa5980b55a7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:21.888450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:21.889076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f38fffc9-dfe7-4a2a-b92a-e4ea98e5df3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:21.890598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 990/1989] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:21 2022] DELETE /identity/v3/projects/fc44f2c4d709473e82a0aa5980b55a7b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:21.895358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45080e-846d-4183-992e-b6594a8a90a7 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:21.895651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45080e-846d-4183-992e-b6594a8a90a7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:21.895934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45080e-846d-4183-992e-b6594a8a90a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:21.912827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-6e45080e-846d-4183-992e-b6594a8a90a7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:21.914032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6e45080e-846d-4183-992e-b6594a8a90a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:21.933987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:21.975030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1000/1990] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:21.982649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc850de-0557-4275-b0cb-6014a9fdac16 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:21.982958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc850de-0557-4275-b0cb-6014a9fdac16 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:21.983262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc850de-0557-4275-b0cb-6014a9fdac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:22.001810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-dbc850de-0557-4275-b0cb-6014a9fdac16 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:22.003015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-dbc850de-0557-4275-b0cb-6014a9fdac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:22.011431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 991/1991] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:22.015506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45080e-846d-4183-992e-b6594a8a90a7 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:22.025757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45080e-846d-4183-992e-b6594a8a90a7 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:22.030375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a4a33-5134-4324-8495-e22f9ad574fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:22.031061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a4a33-5134-4324-8495-e22f9ad574fb admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:22.031348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a4a33-5134-4324-8495-e22f9ad574fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:22.031629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a4a33-5134-4324-8495-e22f9ad574fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:22.032359 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a4a33-5134-4324-8495-e22f9ad574fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:22.032670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a4a33-5134-4324-8495-e22f9ad574fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:22.034268 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a4a33-5134-4324-8495-e22f9ad574fb admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:22.045668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1001/1992] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:14:22.049622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc850de-0557-4275-b0cb-6014a9fdac16 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:22.059077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc850de-0557-4275-b0cb-6014a9fdac16 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:22.063843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38437bb4-aad0-493f-9000-7113104a172e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:22.065665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38437bb4-aad0-493f-9000-7113104a172e admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:22.065665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38437bb4-aad0-493f-9000-7113104a172e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:22.065665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38437bb4-aad0-493f-9000-7113104a172e admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:22.070443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38437bb4-aad0-493f-9000-7113104a172e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:22.073179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38437bb4-aad0-493f-9000-7113104a172e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:22.093341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:22.111813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 992/1993] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:22 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:22.115954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a4a33-5134-4324-8495-e22f9ad574fb admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:22.125833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a4a33-5134-4324-8495-e22f9ad574fb admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:22.183126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e904bd-2608-416b-becd-e7f1757f6495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:22.183923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e904bd-2608-416b-becd-e7f1757f6495 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:22.184232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e904bd-2608-416b-becd-e7f1757f6495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:22.184511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e904bd-2608-416b-becd-e7f1757f6495 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:22.185214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e904bd-2608-416b-becd-e7f1757f6495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:22.187210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e904bd-2608-416b-becd-e7f1757f6495 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:22.214460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1002/1994] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:22 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:22.219771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38437bb4-aad0-493f-9000-7113104a172e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:22.231062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38437bb4-aad0-493f-9000-7113104a172e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:22.236913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799fd91-6ad1-4007-9b23-ed2b54f5fc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:22.237683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799fd91-6ad1-4007-9b23-ed2b54f5fc27 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:22.238117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799fd91-6ad1-4007-9b23-ed2b54f5fc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:22.238466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799fd91-6ad1-4007-9b23-ed2b54f5fc27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:22.239306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799fd91-6ad1-4007-9b23-ed2b54f5fc27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:22.241110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799fd91-6ad1-4007-9b23-ed2b54f5fc27 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:22.258808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 993/1995] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:22.262881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e904bd-2608-416b-becd-e7f1757f6495 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:22.274980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e904bd-2608-416b-becd-e7f1757f6495 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:22.287181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3318d9d-7e51-4384-9c23-08184d3d20d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:22.287181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3318d9d-7e51-4384-9c23-08184d3d20d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:22.287181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3318d9d-7e51-4384-9c23-08184d3d20d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:22.287181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3318d9d-7e51-4384-9c23-08184d3d20d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:22.287704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3318d9d-7e51-4384-9c23-08184d3d20d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:22.287704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3318d9d-7e51-4384-9c23-08184d3d20d1 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:22.307098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1003/1996] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:14:22.308619 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799fd91-6ad1-4007-9b23-ed2b54f5fc27 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:22.319338 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799fd91-6ad1-4007-9b23-ed2b54f5fc27 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:22.327022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24745b0-f093-4c14-9162-537f611f289b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:22.328370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24745b0-f093-4c14-9162-537f611f289b admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:22.328577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24745b0-f093-4c14-9162-537f611f289b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:22.328856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24745b0-f093-4c14-9162-537f611f289b admin admin] PATH_INFO: `/v3/projects/2b9237917037490cb7666828b9b9a3b9/users/487bdd44747844b1b5630bbec7c17925/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:22.344957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24745b0-f093-4c14-9162-537f611f289b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b9237917037490cb7666828b9b9a3b9, user_id=487bdd44747844b1b5630bbec7c17925, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:22.348244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24745b0-f093-4c14-9162-537f611f289b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:22.362546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 994/1997] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:22 2022] PUT /identity/v3/projects/2b9237917037490cb7666828b9b9a3b9/users/487bdd44747844b1b5630bbec7c17925/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:22.373027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900c23a-1bae-41d5-bc99-616cc191edc3 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:22.373324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900c23a-1bae-41d5-bc99-616cc191edc3 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:22.373609 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900c23a-1bae-41d5-bc99-616cc191edc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:22.401733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c900c23a-1bae-41d5-bc99-616cc191edc3 None None] MFA Rules not processed for user `487bdd44747844b1b5630bbec7c17925`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:22.402984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c900c23a-1bae-41d5-bc99-616cc191edc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:22.471788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1004/1998] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:22.483650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24745b0-f093-4c14-9162-537f611f289b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:22.485351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f24745b0-f093-4c14-9162-537f611f289b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:22.551843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24745b0-f093-4c14-9162-537f611f289b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:22.555648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f13d6-da06-4d8c-971a-e82061e383be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450bcd0f22b840c299d08ce9433d6c85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:22.556295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f13d6-da06-4d8c-971a-e82061e383be service glance] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:22.556582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f13d6-da06-4d8c-971a-e82061e383be service glance] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:22.556863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f13d6-da06-4d8c-971a-e82061e383be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:22.565025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f13d6-da06-4d8c-971a-e82061e383be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:22.566448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f13d6-da06-4d8c-971a-e82061e383be service glance] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:22.575126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 995/1999] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:24.493724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900c23a-1bae-41d5-bc99-616cc191edc3 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:24.495336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c900c23a-1bae-41d5-bc99-616cc191edc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:24.518898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:24.662736 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900c23a-1bae-41d5-bc99-616cc191edc3 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:24.670997 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:24.671859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:24.672192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:24.672410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] PATH_INFO: `/v3/users/06ac081b00184db99f6c0cc0b6a7df35` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:24.673625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ac081b00184db99f6c0cc0b6a7df35)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:24.675976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:24.757257 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ac081b00184db99f6c0cc0b6a7df35'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:24.758138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ac081b00184db99f6c0cc0b6a7df35'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:24.758790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ac081b00184db99f6c0cc0b6a7df35'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:24.773758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ac081b00184db99f6c0cc0b6a7df35'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:24.785341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ac081b00184db99f6c0cc0b6a7df35'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:24.818274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1005/2000] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:24 2022] DELETE /identity/v3/users/06ac081b00184db99f6c0cc0b6a7df35 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:24.819974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742081e3-bd57-4bd9-bae4-0c20b35e308e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:24.820487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742081e3-bd57-4bd9-bae4-0c20b35e308e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:24.820684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742081e3-bd57-4bd9-bae4-0c20b35e308e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:24.838534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-742081e3-bd57-4bd9-bae4-0c20b35e308e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:24.839778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-742081e3-bd57-4bd9-bae4-0c20b35e308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:24.920581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 996/2001] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:24.930554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:24.932064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:24.998985 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcaea05-84b7-4f1d-a39c-11e57a9cbb4a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:25.002895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fe1a4-b1cc-4708-ae0e-43aa5bb670fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:25.012044 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fe1a4-b1cc-4708-ae0e-43aa5bb670fd service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:25.012433 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fe1a4-b1cc-4708-ae0e-43aa5bb670fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:25.012730 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fe1a4-b1cc-4708-ae0e-43aa5bb670fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:25.028735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fe1a4-b1cc-4708-ae0e-43aa5bb670fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:25.030301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fe1a4-b1cc-4708-ae0e-43aa5bb670fd service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:25.040865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1006/2002] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:14:25.337751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742081e3-bd57-4bd9-bae4-0c20b35e308e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:25.339288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-742081e3-bd57-4bd9-bae4-0c20b35e308e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:25.386163 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742081e3-bd57-4bd9-bae4-0c20b35e308e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:25.401463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:25.404458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:25.405014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:25.405365 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] PATH_INFO: `/v3/projects/a405d9a1cb1b497f8a4187245cf5cab4` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:25.410648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a405d9a1cb1b497f8a4187245cf5cab4)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:25.414355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:25.499573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a405d9a1cb1b497f8a4187245cf5cab4', 'request_id': 'req-6f0329b6-0f9e-4e86-aaad-359ce630da74'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:25.510232 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a405d9a1cb1b497f8a4187245cf5cab4', 'request_id': 'req-6f0329b6-0f9e-4e86-aaad-359ce630da74'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:25.512063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:25.522018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a405d9a1cb1b497f8a4187245cf5cab4', 'request_id': 'req-6f0329b6-0f9e-4e86-aaad-359ce630da74'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:25.533824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] The token cache is being invalidate because project a405d9a1cb1b497f8a4187245cf5cab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:25.534125 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:25.535411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:25.537203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 997/2003] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:25 2022] DELETE /identity/v3/projects/a405d9a1cb1b497f8a4187245cf5cab4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:25.541089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fe1a4-b1cc-4708-ae0e-43aa5bb670fd service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:25.551200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fe1a4-b1cc-4708-ae0e-43aa5bb670fd service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:25.601586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:25.620145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:25.621155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:25.621504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:25.621789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] PATH_INFO: `/v3/users/016920b63b884497806556a3d7f19f4a` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:25.623273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=016920b63b884497806556a3d7f19f4a)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:25.625381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:25.673115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016920b63b884497806556a3d7f19f4a'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:25.673786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016920b63b884497806556a3d7f19f4a'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:25.674382 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016920b63b884497806556a3d7f19f4a'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:25.681308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016920b63b884497806556a3d7f19f4a'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:25.688641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016920b63b884497806556a3d7f19f4a'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:25.705894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1007/2004] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:25 2022] DELETE /identity/v3/users/016920b63b884497806556a3d7f19f4a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:26.367658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:26.370018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:26.403837 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0329b6-0f9e-4e86-aaad-359ce630da74 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:26.504731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:26.505781 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:26.506069 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:26.506347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] PATH_INFO: `/v3/projects/b2ab964c6bb246aebfa9e5f1ab03ec19` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:26.507597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ab964c6bb246aebfa9e5f1ab03ec19)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:26.509540 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:26.562588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ab964c6bb246aebfa9e5f1ab03ec19', 'request_id': 'req-641935f0-6d37-4abc-9b98-2e81d03f1bfe'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:26.571366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ab964c6bb246aebfa9e5f1ab03ec19', 'request_id': 'req-641935f0-6d37-4abc-9b98-2e81d03f1bfe'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:26.578019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ab964c6bb246aebfa9e5f1ab03ec19', 'request_id': 'req-641935f0-6d37-4abc-9b98-2e81d03f1bfe'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:26.590543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] The token cache is being invalidate because project b2ab964c6bb246aebfa9e5f1ab03ec19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:26.590759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:26.591448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:26.593359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 998/2005] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:26 2022] DELETE /identity/v3/projects/b2ab964c6bb246aebfa9e5f1ab03ec19 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:26.597827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:26.599351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:26.612543 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4cbba-1231-47e7-9a97-560fe84dfed6 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:26.690501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:26.691475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:26.691766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:26.692046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] PATH_INFO: `/v3/users/b0ecebb9635940dabf9a6e67fb06edc0` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:26.693242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0ecebb9635940dabf9a6e67fb06edc0)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:26.695473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:26.749959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0ecebb9635940dabf9a6e67fb06edc0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:26.750641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0ecebb9635940dabf9a6e67fb06edc0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:26.751343 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0ecebb9635940dabf9a6e67fb06edc0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:26.759574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0ecebb9635940dabf9a6e67fb06edc0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:26.767635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b1ec2ff7-ca6c-49b9-a4df-3bb656127041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0ecebb9635940dabf9a6e67fb06edc0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:26.769199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:26.790442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1008/2006] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:26 2022] DELETE /identity/v3/users/b0ecebb9635940dabf9a6e67fb06edc0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:27.066603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:27.068252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:27.094832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641935f0-6d37-4abc-9b98-2e81d03f1bfe admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:27.154157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:27.155086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:27.155378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:27.155655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] PATH_INFO: `/v3/projects/50cea8ac2f154c0aab73f64eb25a701c` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:27.156830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] RBAC: Authorizing `identity:delete_project(project_id=50cea8ac2f154c0aab73f64eb25a701c)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:27.158730 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:27.176017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:27.206956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50cea8ac2f154c0aab73f64eb25a701c', 'request_id': 'req-2b143c98-f924-4a4e-a2a3-1eb54853322a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:27.214247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50cea8ac2f154c0aab73f64eb25a701c', 'request_id': 'req-2b143c98-f924-4a4e-a2a3-1eb54853322a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:27.220534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50cea8ac2f154c0aab73f64eb25a701c', 'request_id': 'req-2b143c98-f924-4a4e-a2a3-1eb54853322a'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:27.231082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] The token cache is being invalidate because project 50cea8ac2f154c0aab73f64eb25a701c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:27.231375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:27.232000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2b143c98-f924-4a4e-a2a3-1eb54853322a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:27.233533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 999/2007] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:27 2022] DELETE /identity/v3/projects/50cea8ac2f154c0aab73f64eb25a701c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:27.239569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ce0ca-507e-4676-8728-7f6c604d5d8d None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:27.239869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ce0ca-507e-4676-8728-7f6c604d5d8d None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:27.240155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ce0ca-507e-4676-8728-7f6c604d5d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:27.258569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-778ce0ca-507e-4676-8728-7f6c604d5d8d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:27.259818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-778ce0ca-507e-4676-8728-7f6c604d5d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:27.319599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1009/2008] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:27.327978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff449f9-2f73-4108-9c40-9d6e1c4b00f1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:27.328270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff449f9-2f73-4108-9c40-9d6e1c4b00f1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:27.328575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff449f9-2f73-4108-9c40-9d6e1c4b00f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:27.347939 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7ff449f9-2f73-4108-9c40-9d6e1c4b00f1 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:27.349257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7ff449f9-2f73-4108-9c40-9d6e1c4b00f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:27.361646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1000/2009] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:27.363491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ce0ca-507e-4676-8728-7f6c604d5d8d None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:27.375336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ce0ca-507e-4676-8728-7f6c604d5d8d None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:27.385466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37575e0-4b3b-4bd8-b150-9597769946a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:27.386251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37575e0-4b3b-4bd8-b150-9597769946a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:27.386634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37575e0-4b3b-4bd8-b150-9597769946a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:27.388616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37575e0-4b3b-4bd8-b150-9597769946a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:27.389702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37575e0-4b3b-4bd8-b150-9597769946a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:27.390530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37575e0-4b3b-4bd8-b150-9597769946a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:27.391963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37575e0-4b3b-4bd8-b150-9597769946a3 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:27.409415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1010/2010] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:14:27.414188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff449f9-2f73-4108-9c40-9d6e1c4b00f1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:27.431073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff449f9-2f73-4108-9c40-9d6e1c4b00f1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:27.439197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d0d50-9a85-420c-a278-7c105c3efd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:27.440368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d0d50-9a85-420c-a278-7c105c3efd46 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:27.441283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d0d50-9a85-420c-a278-7c105c3efd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:27.442075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d0d50-9a85-420c-a278-7c105c3efd46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:27.442955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d0d50-9a85-420c-a278-7c105c3efd46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:27.446256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d0d50-9a85-420c-a278-7c105c3efd46 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:27.495090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1001/2011] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:27 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:27.498290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37575e0-4b3b-4bd8-b150-9597769946a3 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:27.512508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37575e0-4b3b-4bd8-b150-9597769946a3 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:27.609733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1faae0-e48b-49c2-85b5-941f95506946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:27.611163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1faae0-e48b-49c2-85b5-941f95506946 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:27.611445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1faae0-e48b-49c2-85b5-941f95506946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:27.612449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1faae0-e48b-49c2-85b5-941f95506946 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:27.613982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1faae0-e48b-49c2-85b5-941f95506946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:27.616588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1faae0-e48b-49c2-85b5-941f95506946 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:27.648520 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1011/2012] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:27 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:27.654829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d0d50-9a85-420c-a278-7c105c3efd46 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:27.665950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d0d50-9a85-420c-a278-7c105c3efd46 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:27.673677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336c57a-0d5b-4899-8f1f-b13e9cf8ebf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:27.674427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c57a-0d5b-4899-8f1f-b13e9cf8ebf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:27.674750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c57a-0d5b-4899-8f1f-b13e9cf8ebf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:27.675001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336c57a-0d5b-4899-8f1f-b13e9cf8ebf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:27.685577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336c57a-0d5b-4899-8f1f-b13e9cf8ebf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:27.687242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336c57a-0d5b-4899-8f1f-b13e9cf8ebf1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:27.707037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1002/2013] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:27.708503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1faae0-e48b-49c2-85b5-941f95506946 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:27.730569 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1faae0-e48b-49c2-85b5-941f95506946 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:27.741210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dddf250-684e-4081-94af-4bb5256518c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:27.742021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddf250-684e-4081-94af-4bb5256518c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:27.742412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddf250-684e-4081-94af-4bb5256518c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:27.742791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dddf250-684e-4081-94af-4bb5256518c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:27.743819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddf250-684e-4081-94af-4bb5256518c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:27.745675 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dddf250-684e-4081-94af-4bb5256518c5 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:27.758751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1012/2014] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:27.763207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336c57a-0d5b-4899-8f1f-b13e9cf8ebf1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:27.775711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336c57a-0d5b-4899-8f1f-b13e9cf8ebf1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:27.781964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cda99-7d64-4fe1-8404-0cea680d39cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:27.782966 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cda99-7d64-4fe1-8404-0cea680d39cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:27.783391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cda99-7d64-4fe1-8404-0cea680d39cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:27.783769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cda99-7d64-4fe1-8404-0cea680d39cb admin admin] PATH_INFO: `/v3/projects/175136880fff4c6a8e90cf4bff3b4871/users/adbeb2ef64d74b01b1296ecd2bdbd761/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:27.806466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212cda99-7d64-4fe1-8404-0cea680d39cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=175136880fff4c6a8e90cf4bff3b4871, user_id=adbeb2ef64d74b01b1296ecd2bdbd761, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:27.813165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212cda99-7d64-4fe1-8404-0cea680d39cb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:27.832394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1003/2015] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:27 2022] PUT /identity/v3/projects/175136880fff4c6a8e90cf4bff3b4871/users/adbeb2ef64d74b01b1296ecd2bdbd761/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:27.843350 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474259b8-7f84-40fc-8a73-42dc0ae853d0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:27.843869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474259b8-7f84-40fc-8a73-42dc0ae853d0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:27.844489 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474259b8-7f84-40fc-8a73-42dc0ae853d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:27.894706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-474259b8-7f84-40fc-8a73-42dc0ae853d0 None None] MFA Rules not processed for user `adbeb2ef64d74b01b1296ecd2bdbd761`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:27.895543 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-474259b8-7f84-40fc-8a73-42dc0ae853d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:27.941744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:27.986348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1013/2016] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:27.990879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cda99-7d64-4fe1-8404-0cea680d39cb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:28.074475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cda99-7d64-4fe1-8404-0cea680d39cb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:28.079359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd830cbd-e119-4c12-bb81-10a7ebfd30f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:28.079990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd830cbd-e119-4c12-bb81-10a7ebfd30f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:28.080278 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd830cbd-e119-4c12-bb81-10a7ebfd30f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:28.080564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd830cbd-e119-4c12-bb81-10a7ebfd30f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:28.081298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd830cbd-e119-4c12-bb81-10a7ebfd30f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:28.083801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd830cbd-e119-4c12-bb81-10a7ebfd30f8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:28.108757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1004/2017] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:27 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:28.112841 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474259b8-7f84-40fc-8a73-42dc0ae853d0 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:28.123683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474259b8-7f84-40fc-8a73-42dc0ae853d0 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:28.179100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b5afe-8f83-4602-91c8-0902ad61efec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:28.179776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b5afe-8f83-4602-91c8-0902ad61efec admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:28.180068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b5afe-8f83-4602-91c8-0902ad61efec admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:28.180352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b5afe-8f83-4602-91c8-0902ad61efec admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:28.181056 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b5afe-8f83-4602-91c8-0902ad61efec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:28.182994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8b5afe-8f83-4602-91c8-0902ad61efec admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:28.205623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1014/2018] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:28 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:28.210268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd830cbd-e119-4c12-bb81-10a7ebfd30f8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:28.220936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd830cbd-e119-4c12-bb81-10a7ebfd30f8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:28.225474 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a76993-8026-446a-bc49-d4f62cbdc876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:28.226131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a76993-8026-446a-bc49-d4f62cbdc876 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:28.226417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a76993-8026-446a-bc49-d4f62cbdc876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:28.226694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a76993-8026-446a-bc49-d4f62cbdc876 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:28.227453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a76993-8026-446a-bc49-d4f62cbdc876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:28.229052 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a76993-8026-446a-bc49-d4f62cbdc876 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:28.239413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1005/2019] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:28.243578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b5afe-8f83-4602-91c8-0902ad61efec admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:28.254243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b5afe-8f83-4602-91c8-0902ad61efec admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:28.259020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6099185-d67b-4b8f-8f06-6763deea0467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:28.259958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6099185-d67b-4b8f-8f06-6763deea0467 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:28.260248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6099185-d67b-4b8f-8f06-6763deea0467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:28.260528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6099185-d67b-4b8f-8f06-6763deea0467 admin admin] PATH_INFO: `/v3/projects/ebd2da3e22d244be8c0f3cccdeb0a794/users/d4e65114434f490e882a6e2b455ec0bf/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:28.275967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6099185-d67b-4b8f-8f06-6763deea0467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd2da3e22d244be8c0f3cccdeb0a794, user_id=d4e65114434f490e882a6e2b455ec0bf, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:28.278690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6099185-d67b-4b8f-8f06-6763deea0467 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:28.291018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1015/2020] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:28 2022] PUT /identity/v3/projects/ebd2da3e22d244be8c0f3cccdeb0a794/users/d4e65114434f490e882a6e2b455ec0bf/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:28.295070 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a76993-8026-446a-bc49-d4f62cbdc876 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:28.304963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a76993-8026-446a-bc49-d4f62cbdc876 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:28.316683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:28.362386 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6fcb1-8c5d-4095-979b-94a45eba100b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:28.363596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6fcb1-8c5d-4095-979b-94a45eba100b admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:28.363879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6fcb1-8c5d-4095-979b-94a45eba100b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:28.364153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6fcb1-8c5d-4095-979b-94a45eba100b admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:28.364871 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6fcb1-8c5d-4095-979b-94a45eba100b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:28.366583 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6fcb1-8c5d-4095-979b-94a45eba100b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:28.377409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1006/2021] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:28.381537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6099185-d67b-4b8f-8f06-6763deea0467 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:28.392144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6099185-d67b-4b8f-8f06-6763deea0467 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:28.397319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35799410-149f-477c-a5f6-e23599e919f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:28.398201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35799410-149f-477c-a5f6-e23599e919f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:28.398494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35799410-149f-477c-a5f6-e23599e919f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:28.398772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35799410-149f-477c-a5f6-e23599e919f8 admin admin] PATH_INFO: `/v3/projects/ebd2da3e22d244be8c0f3cccdeb0a794/users/d4e65114434f490e882a6e2b455ec0bf/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:28.400773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35799410-149f-477c-a5f6-e23599e919f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd2da3e22d244be8c0f3cccdeb0a794, user_id=d4e65114434f490e882a6e2b455ec0bf, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:28.403450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35799410-149f-477c-a5f6-e23599e919f8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:28.414191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1016/2022] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:28 2022] PUT /identity/v3/projects/ebd2da3e22d244be8c0f3cccdeb0a794/users/d4e65114434f490e882a6e2b455ec0bf/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:28.425432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2869c-90d2-4269-90bc-19e03f6394bb None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:28.425906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2869c-90d2-4269-90bc-19e03f6394bb None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:28.426017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2869c-90d2-4269-90bc-19e03f6394bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:28.452286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-22d2869c-90d2-4269-90bc-19e03f6394bb None None] MFA Rules not processed for user `d4e65114434f490e882a6e2b455ec0bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:28.453522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-22d2869c-90d2-4269-90bc-19e03f6394bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:28.520851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1007/2023] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:28.531684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35799410-149f-477c-a5f6-e23599e919f8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:28.533475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-35799410-149f-477c-a5f6-e23599e919f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:28.592803 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35799410-149f-477c-a5f6-e23599e919f8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:28.597754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8177133-3cfb-46bc-b044-21312995dcde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:28.598415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8177133-3cfb-46bc-b044-21312995dcde service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:28.598704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8177133-3cfb-46bc-b044-21312995dcde service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:28.598982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8177133-3cfb-46bc-b044-21312995dcde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:28.650038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:28.667164 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8177133-3cfb-46bc-b044-21312995dcde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:28.668659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8177133-3cfb-46bc-b044-21312995dcde service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:28.678248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1017/2024] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:38.517287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=56 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:38.673664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=8 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:42.725537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2869c-90d2-4269-90bc-19e03f6394bb None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:42.727007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-22d2869c-90d2-4269-90bc-19e03f6394bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:42.744435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d2869c-90d2-4269-90bc-19e03f6394bb None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:42.746813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cbfe9-d0c6-460c-bf77-4b298aa38d24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:42.747605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cbfe9-d0c6-460c-bf77-4b298aa38d24 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:42.748089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cbfe9-d0c6-460c-bf77-4b298aa38d24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:42.748308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cbfe9-d0c6-460c-bf77-4b298aa38d24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:42.758883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cbfe9-d0c6-460c-bf77-4b298aa38d24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:42.760375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2cbfe9-d0c6-460c-bf77-4b298aa38d24 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:42.770972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1008/2025] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:46.356763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8177133-3cfb-46bc-b044-21312995dcde service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:46.358229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e8177133-3cfb-46bc-b044-21312995dcde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:46.430111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8177133-3cfb-46bc-b044-21312995dcde service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:46.436658 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:46.437640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:46.437933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:46.438218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] PATH_INFO: `/v3/users/41a75990afdf40c0a76d7b499b9a35af` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:46.439479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a75990afdf40c0a76d7b499b9a35af)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:46.442737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:46.511418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a75990afdf40c0a76d7b499b9a35af'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:46.512079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a75990afdf40c0a76d7b499b9a35af'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:46.513557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a75990afdf40c0a76d7b499b9a35af'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:46.523069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a75990afdf40c0a76d7b499b9a35af'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:46.532410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a75990afdf40c0a76d7b499b9a35af'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:46.557840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1018/2026] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:46 2022] DELETE /identity/v3/users/41a75990afdf40c0a76d7b499b9a35af => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:46.562345 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c4f8c-a287-4c1e-ba78-37874d3063fa None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:46.562846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c4f8c-a287-4c1e-ba78-37874d3063fa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:46.562979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c4f8c-a287-4c1e-ba78-37874d3063fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:46.581803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ee0c4f8c-a287-4c1e-ba78-37874d3063fa None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:46.583786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ee0c4f8c-a287-4c1e-ba78-37874d3063fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:46.645079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1009/2027] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:46.654699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:46.656152 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:46.713223 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e11d379-08ea-43d9-82c7-a4da585cdd82 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:46.716896 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e353727-01d3-4aa3-a105-31f3e6640ca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:46.717544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e353727-01d3-4aa3-a105-31f3e6640ca0 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:46.717836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e353727-01d3-4aa3-a105-31f3e6640ca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:46.718119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e353727-01d3-4aa3-a105-31f3e6640ca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:46.728164 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e353727-01d3-4aa3-a105-31f3e6640ca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:46.729675 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e353727-01d3-4aa3-a105-31f3e6640ca0 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:46.740335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1019/2028] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:47.001566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c4f8c-a287-4c1e-ba78-37874d3063fa None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:47.003180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ee0c4f8c-a287-4c1e-ba78-37874d3063fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:47.021493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c4f8c-a287-4c1e-ba78-37874d3063fa None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:47.029835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:47.030739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:47.031016 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:47.031319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] PATH_INFO: `/v3/projects/c09e55ce38e347f08cb629c22ee5d16b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:47.032500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c09e55ce38e347f08cb629c22ee5d16b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:47.034931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:47.082438 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c09e55ce38e347f08cb629c22ee5d16b', 'request_id': 'req-25965a14-b8ec-4b27-bd43-7781f9f26f54'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.089571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c09e55ce38e347f08cb629c22ee5d16b', 'request_id': 'req-25965a14-b8ec-4b27-bd43-7781f9f26f54'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.095625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c09e55ce38e347f08cb629c22ee5d16b', 'request_id': 'req-25965a14-b8ec-4b27-bd43-7781f9f26f54'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.106210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] The token cache is being invalidate because project c09e55ce38e347f08cb629c22ee5d16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:47.106503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.107152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.108664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1010/2029] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:47 2022] DELETE /identity/v3/projects/c09e55ce38e347f08cb629c22ee5d16b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:47.112568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e353727-01d3-4aa3-a105-31f3e6640ca0 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:47.122120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e353727-01d3-4aa3-a105-31f3e6640ca0 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:47.139701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:47.179208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:47.180089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:47.180373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:47.180649 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] PATH_INFO: `/v3/users/bb58af1a40e64ed7ba6171751a955538` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:47.181828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb58af1a40e64ed7ba6171751a955538)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:47.183816 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:47.230453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb58af1a40e64ed7ba6171751a955538'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.231129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb58af1a40e64ed7ba6171751a955538'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.231737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb58af1a40e64ed7ba6171751a955538'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.238377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb58af1a40e64ed7ba6171751a955538'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.245129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bb516f5e-9072-4cf2-ad1e-86cc609c13e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb58af1a40e64ed7ba6171751a955538'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.262226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1020/2030] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:47 2022] DELETE /identity/v3/users/bb58af1a40e64ed7ba6171751a955538 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:47.526648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:47.528111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:47.541568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25965a14-b8ec-4b27-bd43-7781f9f26f54 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:47.602293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:47.603198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:47.603486 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:47.603765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] PATH_INFO: `/v3/projects/72fd939407ce4ef9b35cb9b9a8a6d974` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:47.605026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72fd939407ce4ef9b35cb9b9a8a6d974)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:47.606959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:47.619182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:47.659872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72fd939407ce4ef9b35cb9b9a8a6d974', 'request_id': 'req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.667183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72fd939407ce4ef9b35cb9b9a8a6d974', 'request_id': 'req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.673331 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72fd939407ce4ef9b35cb9b9a8a6d974', 'request_id': 'req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.684037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] The token cache is being invalidate because project 72fd939407ce4ef9b35cb9b9a8a6d974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:47.684326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.684956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1f1d9e7f-4bbd-4a5e-ae69-c15b7320aa08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:47.686487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1011/2031] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:47 2022] DELETE /identity/v3/projects/72fd939407ce4ef9b35cb9b9a8a6d974 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:47.691485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e7627-3a99-49b2-a864-2eb59d6cb40c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:47.691952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e7627-3a99-49b2-a864-2eb59d6cb40c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:47.692045 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e7627-3a99-49b2-a864-2eb59d6cb40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:47.709567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c77e7627-3a99-49b2-a864-2eb59d6cb40c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:47.710755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c77e7627-3a99-49b2-a864-2eb59d6cb40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:47.767798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1021/2032] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:47.778457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0215a63-e9fd-4353-9c1c-bab992e929c7 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:47.778684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0215a63-e9fd-4353-9c1c-bab992e929c7 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:47.778961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0215a63-e9fd-4353-9c1c-bab992e929c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:47.797363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e0215a63-e9fd-4353-9c1c-bab992e929c7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:47.798555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e0215a63-e9fd-4353-9c1c-bab992e929c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:47.807011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1012/2033] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:47.811106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e7627-3a99-49b2-a864-2eb59d6cb40c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:47.821217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e7627-3a99-49b2-a864-2eb59d6cb40c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:47.825707 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d95f5-eeac-4e30-b43b-3a63abddc54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:47.826357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d95f5-eeac-4e30-b43b-3a63abddc54f admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:47.826639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d95f5-eeac-4e30-b43b-3a63abddc54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:47.826914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d95f5-eeac-4e30-b43b-3a63abddc54f admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:47.827671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d95f5-eeac-4e30-b43b-3a63abddc54f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:47.827977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d95f5-eeac-4e30-b43b-3a63abddc54f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:47.829596 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d95f5-eeac-4e30-b43b-3a63abddc54f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:47.841662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1022/2034] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:14:47.845890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0215a63-e9fd-4353-9c1c-bab992e929c7 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:47.855811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0215a63-e9fd-4353-9c1c-bab992e929c7 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:47.860555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98aaf8-c352-4c07-a398-aee91f37b953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:47.861204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98aaf8-c352-4c07-a398-aee91f37b953 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:47.861489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98aaf8-c352-4c07-a398-aee91f37b953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:47.861765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98aaf8-c352-4c07-a398-aee91f37b953 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:47.863003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98aaf8-c352-4c07-a398-aee91f37b953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:47.864933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98aaf8-c352-4c07-a398-aee91f37b953 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:47.890422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1013/2035] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:47 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:47.894616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d95f5-eeac-4e30-b43b-3a63abddc54f admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:47.904444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d95f5-eeac-4e30-b43b-3a63abddc54f admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:47.930263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:47.962446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d9e4a-0995-43ed-bdda-a339e20b0394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:47.963152 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d9e4a-0995-43ed-bdda-a339e20b0394 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:47.963444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d9e4a-0995-43ed-bdda-a339e20b0394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:47.963724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d9e4a-0995-43ed-bdda-a339e20b0394 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:47.964418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d9e4a-0995-43ed-bdda-a339e20b0394 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:47.966334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d9e4a-0995-43ed-bdda-a339e20b0394 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:47.985897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1023/2036] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:47 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:47.990540 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98aaf8-c352-4c07-a398-aee91f37b953 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.000745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98aaf8-c352-4c07-a398-aee91f37b953 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.005998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed2fd4-37d9-4bb5-99b9-f13aa1220e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.006678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed2fd4-37d9-4bb5-99b9-f13aa1220e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.006962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed2fd4-37d9-4bb5-99b9-f13aa1220e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.007271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbed2fd4-37d9-4bb5-99b9-f13aa1220e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.008000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed2fd4-37d9-4bb5-99b9-f13aa1220e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.009617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbed2fd4-37d9-4bb5-99b9-f13aa1220e72 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.020238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1014/2037] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:48.024167 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d9e4a-0995-43ed-bdda-a339e20b0394 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.033994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d9e4a-0995-43ed-bdda-a339e20b0394 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.038490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1b555f-7cfb-4ea5-b604-b207aa24231e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.039195 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b555f-7cfb-4ea5-b604-b207aa24231e admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.039483 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b555f-7cfb-4ea5-b604-b207aa24231e admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.039762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1b555f-7cfb-4ea5-b604-b207aa24231e admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.040534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b555f-7cfb-4ea5-b604-b207aa24231e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.042124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1b555f-7cfb-4ea5-b604-b207aa24231e admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.053978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1024/2038] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:14:48.058004 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed2fd4-37d9-4bb5-99b9-f13aa1220e72 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.068359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbed2fd4-37d9-4bb5-99b9-f13aa1220e72 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.072849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fd99d-99e1-43c2-aa59-75147f2c5dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.073717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fd99d-99e1-43c2-aa59-75147f2c5dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.074364 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fd99d-99e1-43c2-aa59-75147f2c5dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.074654 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fd99d-99e1-43c2-aa59-75147f2c5dd4 admin admin] PATH_INFO: `/v3/projects/5c1e631c1d7b48a7a84362929bba9893/users/155309094da346f7aacc29ca87b894e6/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.088975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fd99d-99e1-43c2-aa59-75147f2c5dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c1e631c1d7b48a7a84362929bba9893, user_id=155309094da346f7aacc29ca87b894e6, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.091656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fd99d-99e1-43c2-aa59-75147f2c5dd4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.104773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1015/2039] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:48 2022] PUT /identity/v3/projects/5c1e631c1d7b48a7a84362929bba9893/users/155309094da346f7aacc29ca87b894e6/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:48.115561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb4b2b-0945-4279-97a4-8f125e29bcdb None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.115863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb4b2b-0945-4279-97a4-8f125e29bcdb None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.116149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb4b2b-0945-4279-97a4-8f125e29bcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.142191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4dbb4b2b-0945-4279-97a4-8f125e29bcdb None None] MFA Rules not processed for user `155309094da346f7aacc29ca87b894e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:48.143456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4dbb4b2b-0945-4279-97a4-8f125e29bcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:48.206193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1025/2040] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:48.210724 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fd99d-99e1-43c2-aa59-75147f2c5dd4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.271710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fd99d-99e1-43c2-aa59-75147f2c5dd4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.276300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952431b0-f686-40ae-9d2f-fd9dfca1eac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.276954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952431b0-f686-40ae-9d2f-fd9dfca1eac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.277514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952431b0-f686-40ae-9d2f-fd9dfca1eac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.277799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952431b0-f686-40ae-9d2f-fd9dfca1eac2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.278492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952431b0-f686-40ae-9d2f-fd9dfca1eac2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.280420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952431b0-f686-40ae-9d2f-fd9dfca1eac2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.305404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1016/2041] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:48 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:48.309367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb4b2b-0945-4279-97a4-8f125e29bcdb None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.320499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb4b2b-0945-4279-97a4-8f125e29bcdb None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.375662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104bed-3fde-47bf-9b4b-95d46246944c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.376326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104bed-3fde-47bf-9b4b-95d46246944c admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.376614 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104bed-3fde-47bf-9b4b-95d46246944c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.376894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18104bed-3fde-47bf-9b4b-95d46246944c admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.377590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18104bed-3fde-47bf-9b4b-95d46246944c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.379585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18104bed-3fde-47bf-9b4b-95d46246944c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.400172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1026/2042] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:48 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:48.404409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952431b0-f686-40ae-9d2f-fd9dfca1eac2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.414992 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952431b0-f686-40ae-9d2f-fd9dfca1eac2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.419600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc39bce-53bf-4993-bef6-608e05918cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.420258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc39bce-53bf-4993-bef6-608e05918cca admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.420549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc39bce-53bf-4993-bef6-608e05918cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.420830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc39bce-53bf-4993-bef6-608e05918cca admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.421560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc39bce-53bf-4993-bef6-608e05918cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.423189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc39bce-53bf-4993-bef6-608e05918cca admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.434486 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1017/2043] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:48.438350 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104bed-3fde-47bf-9b4b-95d46246944c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.448805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18104bed-3fde-47bf-9b4b-95d46246944c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.453379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aef7f0-2645-4ea9-9cdb-61de5568f1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.454252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aef7f0-2645-4ea9-9cdb-61de5568f1b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.454538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aef7f0-2645-4ea9-9cdb-61de5568f1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.454818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aef7f0-2645-4ea9-9cdb-61de5568f1b1 admin admin] PATH_INFO: `/v3/projects/42053e03d9e94acab4717750cbcbae53/users/a5b37897ce0c4dae80e2e034dd96aa57/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.469535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aef7f0-2645-4ea9-9cdb-61de5568f1b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42053e03d9e94acab4717750cbcbae53, user_id=a5b37897ce0c4dae80e2e034dd96aa57, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.472766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aef7f0-2645-4ea9-9cdb-61de5568f1b1 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.484785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1027/2044] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:48 2022] PUT /identity/v3/projects/42053e03d9e94acab4717750cbcbae53/users/a5b37897ce0c4dae80e2e034dd96aa57/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:48.489196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc39bce-53bf-4993-bef6-608e05918cca admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.499241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc39bce-53bf-4993-bef6-608e05918cca admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.507105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:48.557359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b18849-cf53-478e-8730-c1580af4f191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.557944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b18849-cf53-478e-8730-c1580af4f191 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.558229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b18849-cf53-478e-8730-c1580af4f191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.558503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b18849-cf53-478e-8730-c1580af4f191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.559252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b18849-cf53-478e-8730-c1580af4f191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.560839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b18849-cf53-478e-8730-c1580af4f191 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.570324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1018/2045] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:48.574431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aef7f0-2645-4ea9-9cdb-61de5568f1b1 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.579887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:48.586604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aef7f0-2645-4ea9-9cdb-61de5568f1b1 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.591720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a98c4-e4c4-46c9-b08a-a8351e659b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.592605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a98c4-e4c4-46c9-b08a-a8351e659b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.592894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a98c4-e4c4-46c9-b08a-a8351e659b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.593173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a98c4-e4c4-46c9-b08a-a8351e659b3d admin admin] PATH_INFO: `/v3/projects/42053e03d9e94acab4717750cbcbae53/users/a5b37897ce0c4dae80e2e034dd96aa57/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.595201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a98c4-e4c4-46c9-b08a-a8351e659b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=42053e03d9e94acab4717750cbcbae53, user_id=a5b37897ce0c4dae80e2e034dd96aa57, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.598477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a98c4-e4c4-46c9-b08a-a8351e659b3d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.609046 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1028/2046] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:48 2022] PUT /identity/v3/projects/42053e03d9e94acab4717750cbcbae53/users/a5b37897ce0c4dae80e2e034dd96aa57/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:48.619845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d6b38-9ba3-4bae-902b-b0229d3ee678 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.620285 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d6b38-9ba3-4bae-902b-b0229d3ee678 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.620418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d6b38-9ba3-4bae-902b-b0229d3ee678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.646838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4d0d6b38-9ba3-4bae-902b-b0229d3ee678 None None] MFA Rules not processed for user `a5b37897ce0c4dae80e2e034dd96aa57`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:48.648098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4d0d6b38-9ba3-4bae-902b-b0229d3ee678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:48.716476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1019/2047] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:48.728192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a98c4-e4c4-46c9-b08a-a8351e659b3d admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.730109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9d8a98c4-e4c4-46c9-b08a-a8351e659b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:48.786605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a98c4-e4c4-46c9-b08a-a8351e659b3d admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.790317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f8477-650a-4ba7-9bfd-a24fc6973adf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.790964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f8477-650a-4ba7-9bfd-a24fc6973adf service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.791289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f8477-650a-4ba7-9bfd-a24fc6973adf service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.791573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f8477-650a-4ba7-9bfd-a24fc6973adf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.800692 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f8477-650a-4ba7-9bfd-a24fc6973adf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.802116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f8477-650a-4ba7-9bfd-a24fc6973adf service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.811657 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1029/2048] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:48 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:48.870973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d6b38-9ba3-4bae-902b-b0229d3ee678 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:48.872455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4d0d6b38-9ba3-4bae-902b-b0229d3ee678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:48.884735 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d6b38-9ba3-4bae-902b-b0229d3ee678 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:48.888341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3a261-e82c-4ce5-8c31-d6b17dc0e34c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:48.888981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3a261-e82c-4ce5-8c31-d6b17dc0e34c service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:48.889266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3a261-e82c-4ce5-8c31-d6b17dc0e34c service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:48.889543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3a261-e82c-4ce5-8c31-d6b17dc0e34c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:48.946608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3a261-e82c-4ce5-8c31-d6b17dc0e34c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:48.948072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3a261-e82c-4ce5-8c31-d6b17dc0e34c service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:48.957211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1020/2049] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:52.209642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f8477-650a-4ba7-9bfd-a24fc6973adf service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:52.270602 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f8477-650a-4ba7-9bfd-a24fc6973adf service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:52.275170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:52.276036 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:52.276322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:52.276599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] PATH_INFO: `/v3/users/155309094da346f7aacc29ca87b894e6` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:52.277782 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=155309094da346f7aacc29ca87b894e6)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:52.279748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:52.323159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '155309094da346f7aacc29ca87b894e6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:52.323746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '155309094da346f7aacc29ca87b894e6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:52.324346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '155309094da346f7aacc29ca87b894e6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:52.331324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '155309094da346f7aacc29ca87b894e6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:52.338639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '155309094da346f7aacc29ca87b894e6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:52.356263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1030/2050] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:52 2022] DELETE /identity/v3/users/155309094da346f7aacc29ca87b894e6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:52.360716 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258ebfc-cbef-4a14-871a-f198fc4b00d9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:52.361229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258ebfc-cbef-4a14-871a-f198fc4b00d9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:52.361325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258ebfc-cbef-4a14-871a-f198fc4b00d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:52.378755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3258ebfc-cbef-4a14-871a-f198fc4b00d9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:52.379976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3258ebfc-cbef-4a14-871a-f198fc4b00d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:52.404976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:52.438741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1021/2051] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:52.448552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:52.450140 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:52.464931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:52.510210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa2de9-4d7d-4069-804f-383eebc253b0 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:52.514024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3b7ca-9dcf-409b-9cb4-14d484f38b1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:52.514668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3b7ca-9dcf-409b-9cb4-14d484f38b1b service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:52.515596 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3b7ca-9dcf-409b-9cb4-14d484f38b1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:52.516398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3b7ca-9dcf-409b-9cb4-14d484f38b1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:52.526231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3b7ca-9dcf-409b-9cb4-14d484f38b1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:52.527749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e3b7ca-9dcf-409b-9cb4-14d484f38b1b service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:52.537899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1031/2052] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:52.811541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258ebfc-cbef-4a14-871a-f198fc4b00d9 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:52.813114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3258ebfc-cbef-4a14-871a-f198fc4b00d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:52.834605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258ebfc-cbef-4a14-871a-f198fc4b00d9 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:52.840215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:52.841096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:52.841363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:52.841633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] PATH_INFO: `/v3/projects/5c1e631c1d7b48a7a84362929bba9893` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:52.842833 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c1e631c1d7b48a7a84362929bba9893)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:52.844788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:52.894347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c1e631c1d7b48a7a84362929bba9893', 'request_id': 'req-849f6d6a-8564-45ac-842d-36289a179ac0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:52.901758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c1e631c1d7b48a7a84362929bba9893', 'request_id': 'req-849f6d6a-8564-45ac-842d-36289a179ac0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:52.908000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c1e631c1d7b48a7a84362929bba9893', 'request_id': 'req-849f6d6a-8564-45ac-842d-36289a179ac0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:52.917972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] The token cache is being invalidate because project 5c1e631c1d7b48a7a84362929bba9893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:52.918259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:52.918876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:52.920419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1022/2053] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:52 2022] DELETE /identity/v3/projects/5c1e631c1d7b48a7a84362929bba9893 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:52.924460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3b7ca-9dcf-409b-9cb4-14d484f38b1b service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:52.934048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3b7ca-9dcf-409b-9cb4-14d484f38b1b service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:52.992157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:52.993078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:52.993413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:52.993770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] PATH_INFO: `/v3/users/a5b37897ce0c4dae80e2e034dd96aa57` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:52.995211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5b37897ce0c4dae80e2e034dd96aa57)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:52.997333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:53.043368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5b37897ce0c4dae80e2e034dd96aa57'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:53.044019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5b37897ce0c4dae80e2e034dd96aa57'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:53.044620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5b37897ce0c4dae80e2e034dd96aa57'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:53.051782 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5b37897ce0c4dae80e2e034dd96aa57'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:53.059427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5b1ae671-5a66-4dc1-b69b-aa9f33a5971b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5b37897ce0c4dae80e2e034dd96aa57'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:53.076752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1032/2054] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:14:52 2022] DELETE /identity/v3/users/a5b37897ce0c4dae80e2e034dd96aa57 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:53.344047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:53.345648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:53.360860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f6d6a-8564-45ac-842d-36289a179ac0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:53.417703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:53.418617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:53.418876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:53.419181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] PATH_INFO: `/v3/projects/42053e03d9e94acab4717750cbcbae53` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:53.420368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] RBAC: Authorizing `identity:delete_project(project_id=42053e03d9e94acab4717750cbcbae53)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:53.422296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:53.468496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42053e03d9e94acab4717750cbcbae53', 'request_id': 'req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:53.475965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42053e03d9e94acab4717750cbcbae53', 'request_id': 'req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:53.477080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:53.483707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42053e03d9e94acab4717750cbcbae53', 'request_id': 'req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:53.494748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] The token cache is being invalidate because project 42053e03d9e94acab4717750cbcbae53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:14:53.495037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:53.495683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3eab3868-cd7e-417d-ae24-c3ad2d9e221d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:14:53.497223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1023/2055] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:14:53 2022] DELETE /identity/v3/projects/42053e03d9e94acab4717750cbcbae53 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:53.502071 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4655b-11b0-4a93-81ff-ea8aedb4bf62 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:53.502554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4655b-11b0-4a93-81ff-ea8aedb4bf62 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:53.502693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4655b-11b0-4a93-81ff-ea8aedb4bf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:53.519619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5cf4655b-11b0-4a93-81ff-ea8aedb4bf62 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:53.520815 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5cf4655b-11b0-4a93-81ff-ea8aedb4bf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:53.549225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:53.579932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1033/2056] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:53.587999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1efc3-3bf4-4b62-9574-c5cb36130ae2 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:53.588297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1efc3-3bf4-4b62-9574-c5cb36130ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:53.588581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1efc3-3bf4-4b62-9574-c5cb36130ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:53.606766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2bd1efc3-3bf4-4b62-9574-c5cb36130ae2 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:53.607999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2bd1efc3-3bf4-4b62-9574-c5cb36130ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:53.616395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1024/2057] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:53.620455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4655b-11b0-4a93-81ff-ea8aedb4bf62 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:53.630100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4655b-11b0-4a93-81ff-ea8aedb4bf62 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:53.634586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa054d-8da6-4c5b-888e-9c2c5b24fe3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:53.635264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa054d-8da6-4c5b-888e-9c2c5b24fe3b admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:53.635551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa054d-8da6-4c5b-888e-9c2c5b24fe3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:53.635825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa054d-8da6-4c5b-888e-9c2c5b24fe3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:53.636546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa054d-8da6-4c5b-888e-9c2c5b24fe3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:14:53.636852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa054d-8da6-4c5b-888e-9c2c5b24fe3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:53.638444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfa054d-8da6-4c5b-888e-9c2c5b24fe3b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:53.649673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1034/2058] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:14:53.653742 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1efc3-3bf4-4b62-9574-c5cb36130ae2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:53.663797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1efc3-3bf4-4b62-9574-c5cb36130ae2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:53.668482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0a488-79f9-4397-acac-2712c9471e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:53.669202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0a488-79f9-4397-acac-2712c9471e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:53.669409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0a488-79f9-4397-acac-2712c9471e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:53.669682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c0a488-79f9-4397-acac-2712c9471e8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:53.670376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0a488-79f9-4397-acac-2712c9471e8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:53.672295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c0a488-79f9-4397-acac-2712c9471e8c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:53.701475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1025/2059] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:53 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:53.705305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa054d-8da6-4c5b-888e-9c2c5b24fe3b admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:53.715527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa054d-8da6-4c5b-888e-9c2c5b24fe3b admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:53.776427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb01bd-8b85-4668-8ca5-7126ed3cbbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:53.777137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb01bd-8b85-4668-8ca5-7126ed3cbbcb admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:53.777455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb01bd-8b85-4668-8ca5-7126ed3cbbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:53.777738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb01bd-8b85-4668-8ca5-7126ed3cbbcb admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:53.778434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb01bd-8b85-4668-8ca5-7126ed3cbbcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:53.780402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb01bd-8b85-4668-8ca5-7126ed3cbbcb admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:53.803992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1035/2060] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:53 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:14:53.805333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0a488-79f9-4397-acac-2712c9471e8c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:53.817027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c0a488-79f9-4397-acac-2712c9471e8c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:53.821898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60d1a7-7327-4cf4-9a9a-35be5cfc5133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:53.822630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60d1a7-7327-4cf4-9a9a-35be5cfc5133 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:53.822934 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60d1a7-7327-4cf4-9a9a-35be5cfc5133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:53.823239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60d1a7-7327-4cf4-9a9a-35be5cfc5133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:53.823965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60d1a7-7327-4cf4-9a9a-35be5cfc5133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:53.826108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60d1a7-7327-4cf4-9a9a-35be5cfc5133 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:53.838696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1026/2061] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:53.842792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb01bd-8b85-4668-8ca5-7126ed3cbbcb admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:53.855350 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb01bd-8b85-4668-8ca5-7126ed3cbbcb admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:53.860851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2ddb1-eaac-4a9f-b57e-9ac13a8a2830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:53.861663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2ddb1-eaac-4a9f-b57e-9ac13a8a2830 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:53.861938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2ddb1-eaac-4a9f-b57e-9ac13a8a2830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:53.862247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2ddb1-eaac-4a9f-b57e-9ac13a8a2830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:53.862981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2ddb1-eaac-4a9f-b57e-9ac13a8a2830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:53.864624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2ddb1-eaac-4a9f-b57e-9ac13a8a2830 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:53.875866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1036/2062] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:53.880222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60d1a7-7327-4cf4-9a9a-35be5cfc5133 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:53.890706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60d1a7-7327-4cf4-9a9a-35be5cfc5133 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:53.895872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322c24d-f6ce-4cf2-8617-f1d1f50808d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:53.897034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322c24d-f6ce-4cf2-8617-f1d1f50808d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:53.897455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322c24d-f6ce-4cf2-8617-f1d1f50808d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:53.898003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322c24d-f6ce-4cf2-8617-f1d1f50808d9 admin admin] PATH_INFO: `/v3/projects/16a06122e53d44faa473f56f09d70ff3/users/dd3cd89d5f4f468bbb35d74f810dec27/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:53.914299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322c24d-f6ce-4cf2-8617-f1d1f50808d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a06122e53d44faa473f56f09d70ff3, user_id=dd3cd89d5f4f468bbb35d74f810dec27, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:53.917072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322c24d-f6ce-4cf2-8617-f1d1f50808d9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:53.930831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1027/2063] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:53 2022] PUT /identity/v3/projects/16a06122e53d44faa473f56f09d70ff3/users/dd3cd89d5f4f468bbb35d74f810dec27/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:53.936109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b38eb-e71f-464c-908c-c5cb3db8ee34 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:53.936408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b38eb-e71f-464c-908c-c5cb3db8ee34 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:53.936715 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b38eb-e71f-464c-908c-c5cb3db8ee34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:53.955308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ea9b38eb-e71f-464c-908c-c5cb3db8ee34 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:53.956544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ea9b38eb-e71f-464c-908c-c5cb3db8ee34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:54.015613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1037/2064] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:54.025288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322c24d-f6ce-4cf2-8617-f1d1f50808d9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:54.026984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e322c24d-f6ce-4cf2-8617-f1d1f50808d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:54.082412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322c24d-f6ce-4cf2-8617-f1d1f50808d9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:54.086086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21caa8-2205-4858-bf27-c6a300aef5f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:54.086722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21caa8-2205-4858-bf27-c6a300aef5f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:54.087005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21caa8-2205-4858-bf27-c6a300aef5f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:54.087586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21caa8-2205-4858-bf27-c6a300aef5f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:54.097003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21caa8-2205-4858-bf27-c6a300aef5f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:54.098423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21caa8-2205-4858-bf27-c6a300aef5f7 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:54.107607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1028/2065] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:14:54.699736 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59068ae-95a9-40e8-883c-05ed327b42d1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:54.700356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59068ae-95a9-40e8-883c-05ed327b42d1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:54.700356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59068ae-95a9-40e8-883c-05ed327b42d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:54.729215 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c59068ae-95a9-40e8-883c-05ed327b42d1 None None] MFA Rules not processed for user `dd3cd89d5f4f468bbb35d74f810dec27`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:54.730472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c59068ae-95a9-40e8-883c-05ed327b42d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:54.765044 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:54.805300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1038/2066] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:54.809929 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21caa8-2205-4858-bf27-c6a300aef5f7 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:54.819706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21caa8-2205-4858-bf27-c6a300aef5f7 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:54.824494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92412a-e70a-4c68-882c-cf6e674d3dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:54.825175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92412a-e70a-4c68-882c-cf6e674d3dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:54.825542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92412a-e70a-4c68-882c-cf6e674d3dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:54.825891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92412a-e70a-4c68-882c-cf6e674d3dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:54.826653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92412a-e70a-4c68-882c-cf6e674d3dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:54.828685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d92412a-e70a-4c68-882c-cf6e674d3dd6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:54.859344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1029/2067] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:14:54 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:54.860508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59068ae-95a9-40e8-883c-05ed327b42d1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:54.871291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59068ae-95a9-40e8-883c-05ed327b42d1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:54.927011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4d260-8313-48b1-b082-320b03fad24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:54.927722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4d260-8313-48b1-b082-320b03fad24c admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:54.928013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4d260-8313-48b1-b082-320b03fad24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:54.928294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4d260-8313-48b1-b082-320b03fad24c admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:54.929039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4d260-8313-48b1-b082-320b03fad24c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:54.930998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad4d260-8313-48b1-b082-320b03fad24c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:54.952851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1039/2068] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:14:54 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:14:54.957262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92412a-e70a-4c68-882c-cf6e674d3dd6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:54.967937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92412a-e70a-4c68-882c-cf6e674d3dd6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:54.972612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c8536-8601-4aa2-92f1-cced700f5ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:54.973229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c8536-8601-4aa2-92f1-cced700f5ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:54.973515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c8536-8601-4aa2-92f1-cced700f5ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:54.973790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562c8536-8601-4aa2-92f1-cced700f5ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:54.974567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c8536-8601-4aa2-92f1-cced700f5ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:54.976131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562c8536-8601-4aa2-92f1-cced700f5ea8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:54.989561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1030/2069] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:54.990678 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4d260-8313-48b1-b082-320b03fad24c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:55.003316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad4d260-8313-48b1-b082-320b03fad24c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:55.008291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1a99a-0a24-4d91-9976-849648daeb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:55.009195 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1a99a-0a24-4d91-9976-849648daeb74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:55.009486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1a99a-0a24-4d91-9976-849648daeb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:55.009772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1a99a-0a24-4d91-9976-849648daeb74 admin admin] PATH_INFO: `/v3/projects/44792fb74fe345d19680430a2f3a909f/users/1d9ab91875cd49e3a10ac52dbd35b2f0/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:55.025603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1a99a-0a24-4d91-9976-849648daeb74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44792fb74fe345d19680430a2f3a909f, user_id=1d9ab91875cd49e3a10ac52dbd35b2f0, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:55.028306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1a99a-0a24-4d91-9976-849648daeb74 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:55.040578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1040/2070] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:54 2022] PUT /identity/v3/projects/44792fb74fe345d19680430a2f3a909f/users/1d9ab91875cd49e3a10ac52dbd35b2f0/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:14:55.044799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c8536-8601-4aa2-92f1-cced700f5ea8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:55.056017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562c8536-8601-4aa2-92f1-cced700f5ea8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:55.103332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:55.114628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aaa93-6fa4-451f-aaae-86da551055d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:55.115308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aaa93-6fa4-451f-aaae-86da551055d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:55.115597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aaa93-6fa4-451f-aaae-86da551055d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:55.115878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aaa93-6fa4-451f-aaae-86da551055d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:55.116608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aaa93-6fa4-451f-aaae-86da551055d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:55.118202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aaa93-6fa4-451f-aaae-86da551055d1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:55.127798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1031/2071] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:14:55.131910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1a99a-0a24-4d91-9976-849648daeb74 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:55.141914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1a99a-0a24-4d91-9976-849648daeb74 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:55.146491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91b18-432e-4777-9742-4d7bfa29e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:55.147394 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91b18-432e-4777-9742-4d7bfa29e86f admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:55.147681 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91b18-432e-4777-9742-4d7bfa29e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:55.147960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91b18-432e-4777-9742-4d7bfa29e86f admin admin] PATH_INFO: `/v3/projects/44792fb74fe345d19680430a2f3a909f/users/1d9ab91875cd49e3a10ac52dbd35b2f0/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:55.149929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb91b18-432e-4777-9742-4d7bfa29e86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=44792fb74fe345d19680430a2f3a909f, user_id=1d9ab91875cd49e3a10ac52dbd35b2f0, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:55.152592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb91b18-432e-4777-9742-4d7bfa29e86f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:55.163090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1041/2072] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:14:55 2022] PUT /identity/v3/projects/44792fb74fe345d19680430a2f3a909f/users/1d9ab91875cd49e3a10ac52dbd35b2f0/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:14:55.759656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf4e34-a236-46be-ba7e-8417cdabd255 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:55.759957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf4e34-a236-46be-ba7e-8417cdabd255 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:55.760243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf4e34-a236-46be-ba7e-8417cdabd255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:55.789787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-12bf4e34-a236-46be-ba7e-8417cdabd255 None None] MFA Rules not processed for user `1d9ab91875cd49e3a10ac52dbd35b2f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:14:55.791146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-12bf4e34-a236-46be-ba7e-8417cdabd255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:55.871612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1032/2073] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:14:55 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:14:55.883852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91b18-432e-4777-9742-4d7bfa29e86f admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:14:55.885626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bfb91b18-432e-4777-9742-4d7bfa29e86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:14:55.942938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91b18-432e-4777-9742-4d7bfa29e86f admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:14:55.946661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9f7c6-5d12-459b-8df5-22de45fd8f81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:14:55.947390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9f7c6-5d12-459b-8df5-22de45fd8f81 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:14:55.947675 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9f7c6-5d12-459b-8df5-22de45fd8f81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:14:55.947950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9f7c6-5d12-459b-8df5-22de45fd8f81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:14:55.985133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:14:56.022355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9f7c6-5d12-459b-8df5-22de45fd8f81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:14:56.024095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9f7c6-5d12-459b-8df5-22de45fd8f81 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:14:56.033389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1042/2074] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:14:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:00.529862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf4e34-a236-46be-ba7e-8417cdabd255 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:00.531340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-12bf4e34-a236-46be-ba7e-8417cdabd255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:00.543415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf4e34-a236-46be-ba7e-8417cdabd255 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:00.546982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d51fc-15a3-4480-8346-ffb036312149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:00.547650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d51fc-15a3-4480-8346-ffb036312149 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:00.547937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d51fc-15a3-4480-8346-ffb036312149 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:00.548224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d51fc-15a3-4480-8346-ffb036312149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:00.557351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d51fc-15a3-4480-8346-ffb036312149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:00.558901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d51fc-15a3-4480-8346-ffb036312149 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:00.569764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1033/2075] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:02.403505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9f7c6-5d12-459b-8df5-22de45fd8f81 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:02.404922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-84a9f7c6-5d12-459b-8df5-22de45fd8f81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:02.469195 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9f7c6-5d12-459b-8df5-22de45fd8f81 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:02.474229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:02.475128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:02.475414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:02.475692 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] PATH_INFO: `/v3/users/adbeb2ef64d74b01b1296ecd2bdbd761` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:02.476875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] RBAC: Authorizing `identity:delete_user(user_id=adbeb2ef64d74b01b1296ecd2bdbd761)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:02.478799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:02.525960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adbeb2ef64d74b01b1296ecd2bdbd761'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:02.526629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adbeb2ef64d74b01b1296ecd2bdbd761'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:02.527257 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adbeb2ef64d74b01b1296ecd2bdbd761'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:02.534453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adbeb2ef64d74b01b1296ecd2bdbd761'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:02.541538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adbeb2ef64d74b01b1296ecd2bdbd761'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:02.560109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1043/2076] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:02 2022] DELETE /identity/v3/users/adbeb2ef64d74b01b1296ecd2bdbd761 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:02.564521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57f39c-6574-4aa6-bd1b-f02566517197 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:02.565021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57f39c-6574-4aa6-bd1b-f02566517197 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:02.565116 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57f39c-6574-4aa6-bd1b-f02566517197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:02.583824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7e57f39c-6574-4aa6-bd1b-f02566517197 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:02.585042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7e57f39c-6574-4aa6-bd1b-f02566517197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:02.644231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1034/2077] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:02.653563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:02.655222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:02.714357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aeee0c-876e-45f5-aab1-c3bb7869f65b admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:02.718277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cb1b0-13bc-4ce0-8586-066129b2fe2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:02.718930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cb1b0-13bc-4ce0-8586-066129b2fe2e service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:02.719251 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cb1b0-13bc-4ce0-8586-066129b2fe2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:02.719540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9cb1b0-13bc-4ce0-8586-066129b2fe2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:02.730570 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9cb1b0-13bc-4ce0-8586-066129b2fe2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:02.732096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9cb1b0-13bc-4ce0-8586-066129b2fe2e service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:02.736208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:02.744059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1044/2078] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:03.044272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57f39c-6574-4aa6-bd1b-f02566517197 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:03.048369 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7e57f39c-6574-4aa6-bd1b-f02566517197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:03.064237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57f39c-6574-4aa6-bd1b-f02566517197 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:03.071162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:03.072325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:03.072805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:03.073092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] PATH_INFO: `/v3/projects/175136880fff4c6a8e90cf4bff3b4871` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:03.074311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=175136880fff4c6a8e90cf4bff3b4871)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:03.076313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:03.152053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175136880fff4c6a8e90cf4bff3b4871', 'request_id': 'req-d7289728-f8bb-41df-a760-0b559d595c65'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.159480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:03.170331 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175136880fff4c6a8e90cf4bff3b4871', 'request_id': 'req-d7289728-f8bb-41df-a760-0b559d595c65'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.179657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175136880fff4c6a8e90cf4bff3b4871', 'request_id': 'req-d7289728-f8bb-41df-a760-0b559d595c65'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.210079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] The token cache is being invalidate because project 175136880fff4c6a8e90cf4bff3b4871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:03.210693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.212030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.215507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1035/2079] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:03 2022] DELETE /identity/v3/projects/175136880fff4c6a8e90cf4bff3b4871 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:03.218143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cb1b0-13bc-4ce0-8586-066129b2fe2e service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:03.228948 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9cb1b0-13bc-4ce0-8586-066129b2fe2e service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:03.287901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:03.288689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:03.288978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:03.289255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] PATH_INFO: `/v3/users/d4e65114434f490e882a6e2b455ec0bf` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:03.290461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4e65114434f490e882a6e2b455ec0bf)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:03.292437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:03.345754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4e65114434f490e882a6e2b455ec0bf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.346530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4e65114434f490e882a6e2b455ec0bf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.347177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4e65114434f490e882a6e2b455ec0bf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.354105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4e65114434f490e882a6e2b455ec0bf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.361500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-18b1b560-1f43-4ee7-aed4-82e8d9debdb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4e65114434f490e882a6e2b455ec0bf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.379913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1045/2080] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:03 2022] DELETE /identity/v3/users/d4e65114434f490e882a6e2b455ec0bf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:03.699818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:03.701384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:03.723403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7289728-f8bb-41df-a760-0b559d595c65 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:03.779168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:03.780033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:03.780314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:03.780586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] PATH_INFO: `/v3/projects/ebd2da3e22d244be8c0f3cccdeb0a794` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:03.781766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd2da3e22d244be8c0f3cccdeb0a794)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:03.783740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:03.828596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd2da3e22d244be8c0f3cccdeb0a794', 'request_id': 'req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.841235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd2da3e22d244be8c0f3cccdeb0a794', 'request_id': 'req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.847726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd2da3e22d244be8c0f3cccdeb0a794', 'request_id': 'req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.857867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] The token cache is being invalidate because project ebd2da3e22d244be8c0f3cccdeb0a794 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:03.858159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.858789 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0a03a13a-bcc4-4748-a6c1-1aa4958acb3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:03.860409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1036/2081] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:03 2022] DELETE /identity/v3/projects/ebd2da3e22d244be8c0f3cccdeb0a794 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:03.867825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a36a61-384f-43be-9cc8-8a2a605a22ee None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:03.868323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a36a61-384f-43be-9cc8-8a2a605a22ee None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:03.868476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a36a61-384f-43be-9cc8-8a2a605a22ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:03.886426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-43a36a61-384f-43be-9cc8-8a2a605a22ee None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:03.887650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-43a36a61-384f-43be-9cc8-8a2a605a22ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:03.946212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1046/2082] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:03.954372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f99767-6c50-4328-9a60-fab0d7f1e32e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:03.954663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f99767-6c50-4328-9a60-fab0d7f1e32e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:03.954945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f99767-6c50-4328-9a60-fab0d7f1e32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:03.973944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e1f99767-6c50-4328-9a60-fab0d7f1e32e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:03.975182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e1f99767-6c50-4328-9a60-fab0d7f1e32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:03.982995 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1037/2083] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:03.987661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a36a61-384f-43be-9cc8-8a2a605a22ee None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:03.998351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a36a61-384f-43be-9cc8-8a2a605a22ee None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:04.003125 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ff349-c868-407c-9e89-92930ee0ecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:04.003786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ff349-c868-407c-9e89-92930ee0ecb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:04.004073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ff349-c868-407c-9e89-92930ee0ecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:04.004357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ff349-c868-407c-9e89-92930ee0ecb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:04.005082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ff349-c868-407c-9e89-92930ee0ecb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:15:04.005391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ff349-c868-407c-9e89-92930ee0ecb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:04.006994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ff349-c868-407c-9e89-92930ee0ecb7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:04.018473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1047/2084] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:15:04.022798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f99767-6c50-4328-9a60-fab0d7f1e32e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:04.032465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f99767-6c50-4328-9a60-fab0d7f1e32e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:04.037606 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be8811-99fe-4eb7-ae19-5b3b028d4cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:04.038254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be8811-99fe-4eb7-ae19-5b3b028d4cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:04.038540 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be8811-99fe-4eb7-ae19-5b3b028d4cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:04.038824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be8811-99fe-4eb7-ae19-5b3b028d4cb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:04.039584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be8811-99fe-4eb7-ae19-5b3b028d4cb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:04.041489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be8811-99fe-4eb7-ae19-5b3b028d4cb1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:04.067214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1038/2085] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:15:04 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:04.071370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ff349-c868-407c-9e89-92930ee0ecb7 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:04.077568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:04.081724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ff349-c868-407c-9e89-92930ee0ecb7 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:04.138844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1d49e-9b21-45cb-983c-ec43d28d8273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:04.139532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1d49e-9b21-45cb-983c-ec43d28d8273 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:04.139820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1d49e-9b21-45cb-983c-ec43d28d8273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:04.140102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1d49e-9b21-45cb-983c-ec43d28d8273 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:04.140794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1d49e-9b21-45cb-983c-ec43d28d8273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:04.142700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1d49e-9b21-45cb-983c-ec43d28d8273 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:04.162173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1048/2086] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:15:04 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:04.166260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be8811-99fe-4eb7-ae19-5b3b028d4cb1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:04.178444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be8811-99fe-4eb7-ae19-5b3b028d4cb1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:04.183072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fa05e-2f99-4e1c-98c0-b0c10c43a177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:04.183732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fa05e-2f99-4e1c-98c0-b0c10c43a177 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:04.184017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fa05e-2f99-4e1c-98c0-b0c10c43a177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:04.184295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1fa05e-2f99-4e1c-98c0-b0c10c43a177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:04.185017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fa05e-2f99-4e1c-98c0-b0c10c43a177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:04.186594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1fa05e-2f99-4e1c-98c0-b0c10c43a177 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:04.197813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1039/2087] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:04.201823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1d49e-9b21-45cb-983c-ec43d28d8273 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:04.213161 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1d49e-9b21-45cb-983c-ec43d28d8273 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:04.217855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a92e2-39db-45d9-b884-c460e598c922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:04.218523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a92e2-39db-45d9-b884-c460e598c922 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:04.218813 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a92e2-39db-45d9-b884-c460e598c922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:04.219123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a92e2-39db-45d9-b884-c460e598c922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:04.219861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a92e2-39db-45d9-b884-c460e598c922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:04.221479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42a92e2-39db-45d9-b884-c460e598c922 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:04.231615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1049/2088] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:04.235938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fa05e-2f99-4e1c-98c0-b0c10c43a177 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:04.246567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1fa05e-2f99-4e1c-98c0-b0c10c43a177 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:04.251255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c3dc8-1ec3-4af9-a4d4-c788025c951f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:04.252140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c3dc8-1ec3-4af9-a4d4-c788025c951f admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:04.252421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c3dc8-1ec3-4af9-a4d4-c788025c951f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:04.253187 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79c3dc8-1ec3-4af9-a4d4-c788025c951f admin admin] PATH_INFO: `/v3/projects/0ff25c5cca1d44179e2247b8cfb00aff/users/bafc322cedbe4f3aa24ff8567a2767d4/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:04.268074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c3dc8-1ec3-4af9-a4d4-c788025c951f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ff25c5cca1d44179e2247b8cfb00aff, user_id=bafc322cedbe4f3aa24ff8567a2767d4, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:04.271428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79c3dc8-1ec3-4af9-a4d4-c788025c951f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:04.285553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1040/2089] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:15:04 2022] PUT /identity/v3/projects/0ff25c5cca1d44179e2247b8cfb00aff/users/bafc322cedbe4f3aa24ff8567a2767d4/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:04.297421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fda901-3a50-495d-98cf-82d6b3798224 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:04.297721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fda901-3a50-495d-98cf-82d6b3798224 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:04.298085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fda901-3a50-495d-98cf-82d6b3798224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:04.324024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-62fda901-3a50-495d-98cf-82d6b3798224 None None] MFA Rules not processed for user `bafc322cedbe4f3aa24ff8567a2767d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:04.325220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-62fda901-3a50-495d-98cf-82d6b3798224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:04.391385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1050/2090] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:04.402562 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c3dc8-1ec3-4af9-a4d4-c788025c951f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:04.404657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f79c3dc8-1ec3-4af9-a4d4-c788025c951f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:04.415915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:04.462713 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79c3dc8-1ec3-4af9-a4d4-c788025c951f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:04.466378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60609d49-698b-423c-812f-b95235dda948 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:04.467018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60609d49-698b-423c-812f-b95235dda948 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:04.467335 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60609d49-698b-423c-812f-b95235dda948 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:04.467615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60609d49-698b-423c-812f-b95235dda948 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:04.476527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60609d49-698b-423c-812f-b95235dda948 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:04.478013 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60609d49-698b-423c-812f-b95235dda948 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:04.487351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1041/2091] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:09.986561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fda901-3a50-495d-98cf-82d6b3798224 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:09.988622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-62fda901-3a50-495d-98cf-82d6b3798224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:10.054860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fda901-3a50-495d-98cf-82d6b3798224 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:10.059611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:10.060549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:10.060919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:10.061199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] PATH_INFO: `/v3/users/487bdd44747844b1b5630bbec7c17925` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:10.062391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=487bdd44747844b1b5630bbec7c17925)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:10.065031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:10.081225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:10.133605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '487bdd44747844b1b5630bbec7c17925'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:10.134363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '487bdd44747844b1b5630bbec7c17925'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:10.135067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '487bdd44747844b1b5630bbec7c17925'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:10.142731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '487bdd44747844b1b5630bbec7c17925'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:10.150761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '487bdd44747844b1b5630bbec7c17925'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:10.171381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1051/2092] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:09 2022] DELETE /identity/v3/users/487bdd44747844b1b5630bbec7c17925 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:10.176733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8870b-0cee-49b1-a8cf-f40d55502db5 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:10.177240 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8870b-0cee-49b1-a8cf-f40d55502db5 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:10.177335 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c8870b-0cee-49b1-a8cf-f40d55502db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:10.194990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-25c8870b-0cee-49b1-a8cf-f40d55502db5 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:10.196246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-25c8870b-0cee-49b1-a8cf-f40d55502db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:10.269390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1042/2093] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:15:10.280523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:10.282041 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:10.346580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5caaf94-a60b-4bfd-8bee-750c4afafcd2 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:10.350445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922e39d-4673-466e-bd2a-fb56bbbe5b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:10.351221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922e39d-4673-466e-bd2a-fb56bbbe5b71 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:10.351586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922e39d-4673-466e-bd2a-fb56bbbe5b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:10.352089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922e39d-4673-466e-bd2a-fb56bbbe5b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:10.362528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922e39d-4673-466e-bd2a-fb56bbbe5b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:10.364088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5922e39d-4673-466e-bd2a-fb56bbbe5b71 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:10.374489 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1052/2094] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:10.645404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8870b-0cee-49b1-a8cf-f40d55502db5 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:10.647200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-25c8870b-0cee-49b1-a8cf-f40d55502db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:10.664004 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c8870b-0cee-49b1-a8cf-f40d55502db5 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:10.686999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:10.686999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:10.686999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:10.686999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] PATH_INFO: `/v3/projects/2b9237917037490cb7666828b9b9a3b9` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:10.687318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b9237917037490cb7666828b9b9a3b9)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:10.687318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:10.784788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b9237917037490cb7666828b9b9a3b9', 'request_id': 'req-e0a4a4a0-930b-47d6-8442-895ae731df3e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:10.797618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b9237917037490cb7666828b9b9a3b9', 'request_id': 'req-e0a4a4a0-930b-47d6-8442-895ae731df3e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:10.809068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b9237917037490cb7666828b9b9a3b9', 'request_id': 'req-e0a4a4a0-930b-47d6-8442-895ae731df3e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:10.824737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] The token cache is being invalidate because project 2b9237917037490cb7666828b9b9a3b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:10.825241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:10.826920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e0a4a4a0-930b-47d6-8442-895ae731df3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:10.832274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1043/2095] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:10 2022] DELETE /identity/v3/projects/2b9237917037490cb7666828b9b9a3b9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:10.843432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ff316-d998-4d59-ab7a-dd0169d41969 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:10.843432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ff316-d998-4d59-ab7a-dd0169d41969 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:10.843432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ff316-d998-4d59-ab7a-dd0169d41969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:10.868210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-cc5ff316-d998-4d59-ab7a-dd0169d41969 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:10.868210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-cc5ff316-d998-4d59-ab7a-dd0169d41969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:10.948495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1053/2096] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:10.958269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53b14f-f391-48a8-8593-e5dd10fc4d64 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:10.958909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53b14f-f391-48a8-8593-e5dd10fc4d64 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:10.959022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53b14f-f391-48a8-8593-e5dd10fc4d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:10.983177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ca53b14f-f391-48a8-8593-e5dd10fc4d64 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:10.984868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ca53b14f-f391-48a8-8593-e5dd10fc4d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:10.994906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1044/2097] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:15:10.999835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ff316-d998-4d59-ab7a-dd0169d41969 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:11.011823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ff316-d998-4d59-ab7a-dd0169d41969 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:11.016690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2003e-6d67-4e54-b724-d5d309eec937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:11.016920 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2003e-6d67-4e54-b724-d5d309eec937 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:11.017174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2003e-6d67-4e54-b724-d5d309eec937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:11.017460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2003e-6d67-4e54-b724-d5d309eec937 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:11.018250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2003e-6d67-4e54-b724-d5d309eec937 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:15:11.018559 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2003e-6d67-4e54-b724-d5d309eec937 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:11.027386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2003e-6d67-4e54-b724-d5d309eec937 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:11.051674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1054/2098] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:15:11.055962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53b14f-f391-48a8-8593-e5dd10fc4d64 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:11.069385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53b14f-f391-48a8-8593-e5dd10fc4d64 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:11.076937 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4b311-474a-4d8f-9fd6-6d3c66bdd078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:11.077874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4b311-474a-4d8f-9fd6-6d3c66bdd078 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:11.078346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4b311-474a-4d8f-9fd6-6d3c66bdd078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:11.078773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4b311-474a-4d8f-9fd6-6d3c66bdd078 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:11.079737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4b311-474a-4d8f-9fd6-6d3c66bdd078 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:11.082748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4b311-474a-4d8f-9fd6-6d3c66bdd078 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:11.114242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1045/2099] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:15:11 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:11.118770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2003e-6d67-4e54-b724-d5d309eec937 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:11.129704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2003e-6d67-4e54-b724-d5d309eec937 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:11.156159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:11.225262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba5a03-5a10-436a-9225-d0c7fdbeb2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:11.225937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba5a03-5a10-436a-9225-d0c7fdbeb2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:11.226230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba5a03-5a10-436a-9225-d0c7fdbeb2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:11.226514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ba5a03-5a10-436a-9225-d0c7fdbeb2d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:11.227248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba5a03-5a10-436a-9225-d0c7fdbeb2d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:11.229200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ba5a03-5a10-436a-9225-d0c7fdbeb2d8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:11.251356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1055/2100] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:15:11 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:11.255963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4b311-474a-4d8f-9fd6-6d3c66bdd078 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:11.266512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4b311-474a-4d8f-9fd6-6d3c66bdd078 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:11.271218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b975a3-d185-4a7d-9f56-6b5d0fef4001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:11.271884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b975a3-d185-4a7d-9f56-6b5d0fef4001 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:11.272167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b975a3-d185-4a7d-9f56-6b5d0fef4001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:11.272455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b975a3-d185-4a7d-9f56-6b5d0fef4001 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:11.273214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b975a3-d185-4a7d-9f56-6b5d0fef4001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:11.275149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b975a3-d185-4a7d-9f56-6b5d0fef4001 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:11.276477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:11.289783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1046/2101] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:11.291121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba5a03-5a10-436a-9225-d0c7fdbeb2d8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:11.304856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ba5a03-5a10-436a-9225-d0c7fdbeb2d8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:11.309720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8a3ea-3404-466d-b7a7-8fa1766f16dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:11.310391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8a3ea-3404-466d-b7a7-8fa1766f16dd admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:11.310679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8a3ea-3404-466d-b7a7-8fa1766f16dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:11.310965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8a3ea-3404-466d-b7a7-8fa1766f16dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:11.311748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8a3ea-3404-466d-b7a7-8fa1766f16dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:11.313373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8a3ea-3404-466d-b7a7-8fa1766f16dd admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:11.328273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1056/2102] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:15:11.329453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b975a3-d185-4a7d-9f56-6b5d0fef4001 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:11.340051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b975a3-d185-4a7d-9f56-6b5d0fef4001 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:11.344678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cdf8e-be0b-4581-aa6d-e086f2557f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:11.345561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cdf8e-be0b-4581-aa6d-e086f2557f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:11.345846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cdf8e-be0b-4581-aa6d-e086f2557f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:11.346157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cdf8e-be0b-4581-aa6d-e086f2557f53 admin admin] PATH_INFO: `/v3/projects/412f781a4ac24d06ab713b8a63383d5a/users/c8a3657a1e3a4e40bdbb9f970cf822c4/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:11.362153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cdf8e-be0b-4581-aa6d-e086f2557f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412f781a4ac24d06ab713b8a63383d5a, user_id=c8a3657a1e3a4e40bdbb9f970cf822c4, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:11.365050 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cdf8e-be0b-4581-aa6d-e086f2557f53 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:11.379063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1047/2103] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:15:11 2022] PUT /identity/v3/projects/412f781a4ac24d06ab713b8a63383d5a/users/c8a3657a1e3a4e40bdbb9f970cf822c4/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:11.388845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0daa97-aeae-46b8-897e-ce71fc28a882 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:11.389143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0daa97-aeae-46b8-897e-ce71fc28a882 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:11.389431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0daa97-aeae-46b8-897e-ce71fc28a882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:11.417979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0e0daa97-aeae-46b8-897e-ce71fc28a882 None None] MFA Rules not processed for user `c8a3657a1e3a4e40bdbb9f970cf822c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:11.419444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0e0daa97-aeae-46b8-897e-ce71fc28a882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:11.507436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1057/2104] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:11.523003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cdf8e-be0b-4581-aa6d-e086f2557f53 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:11.524852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b50cdf8e-be0b-4581-aa6d-e086f2557f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:11.590756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cdf8e-be0b-4581-aa6d-e086f2557f53 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:11.594580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6c7a2-caea-4b9d-8935-f734039dc7d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:11.595263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6c7a2-caea-4b9d-8935-f734039dc7d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:11.595555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6c7a2-caea-4b9d-8935-f734039dc7d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:11.596003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6c7a2-caea-4b9d-8935-f734039dc7d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:11.606957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6c7a2-caea-4b9d-8935-f734039dc7d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:11.608455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6c7a2-caea-4b9d-8935-f734039dc7d5 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:11.617879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1048/2105] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:14.355774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9c58d-9ca1-4f1d-a83f-56f84a07615a None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:14.358073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9c58d-9ca1-4f1d-a83f-56f84a07615a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:14.358598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9c58d-9ca1-4f1d-a83f-56f84a07615a None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:14.360500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1058/2106] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:15:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:15:14.369990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead108ad-da76-402a-bbb1-333ae0615741 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:14.370498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead108ad-da76-402a-bbb1-333ae0615741 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:14.371089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead108ad-da76-402a-bbb1-333ae0615741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:14.401603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ead108ad-da76-402a-bbb1-333ae0615741 None None] MFA Rules not processed for user `c68f1297771e4c03a897dd1f0ac21cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:14.402983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ead108ad-da76-402a-bbb1-333ae0615741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:14.480821 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9c58d-9ca1-4f1d-a83f-56f84a07615a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:14.483475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f8e9c58d-9ca1-4f1d-a83f-56f84a07615a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:14.500657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1049/2107] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:15:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:14.512451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead108ad-da76-402a-bbb1-333ae0615741 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:14.515700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ead108ad-da76-402a-bbb1-333ae0615741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:14.531478 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead108ad-da76-402a-bbb1-333ae0615741 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:14.537620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa3394-1863-423f-b7a7-b9b6861afb63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:14.538508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa3394-1863-423f-b7a7-b9b6861afb63 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:14.539294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa3394-1863-423f-b7a7-b9b6861afb63 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:14.539686 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa3394-1863-423f-b7a7-b9b6861afb63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:14.552581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa3394-1863-423f-b7a7-b9b6861afb63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:14.555242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa3394-1863-423f-b7a7-b9b6861afb63 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:14.572337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1050/2108] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:15:14.584074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9c58d-9ca1-4f1d-a83f-56f84a07615a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:14.592397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:14.593643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:14.594027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:14.594410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] PATH_INFO: `/v3/users/ba0f3ef2b024464a9c3e71f5d639ca69` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:14.596809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba0f3ef2b024464a9c3e71f5d639ca69)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:14.598909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:14.649927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:14.667088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba0f3ef2b024464a9c3e71f5d639ca69'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:14.667881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba0f3ef2b024464a9c3e71f5d639ca69'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:14.669109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba0f3ef2b024464a9c3e71f5d639ca69'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:14.678960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba0f3ef2b024464a9c3e71f5d639ca69'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:14.687480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b70a5cec-d966-47d3-b500-a9e837ac15e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba0f3ef2b024464a9c3e71f5d639ca69'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:14.711447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1059/2109] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:14 2022] DELETE /identity/v3/users/ba0f3ef2b024464a9c3e71f5d639ca69 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:14.942200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa3394-1863-423f-b7a7-b9b6861afb63 service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:14.962192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa3394-1863-423f-b7a7-b9b6861afb63 service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:15.026210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:15.035896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:15.037076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.037618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.038094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] PATH_INFO: `/v3/projects/a0ccca42ac0a4113bba224c8d076dd81` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.039411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ccca42ac0a4113bba224c8d076dd81)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:15.041506 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:15.088801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ccca42ac0a4113bba224c8d076dd81', 'request_id': 'req-5106facd-afaf-477e-aa06-7b29a9f22fae'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:15.096622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ccca42ac0a4113bba224c8d076dd81', 'request_id': 'req-5106facd-afaf-477e-aa06-7b29a9f22fae'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:15.103112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ccca42ac0a4113bba224c8d076dd81', 'request_id': 'req-5106facd-afaf-477e-aa06-7b29a9f22fae'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:15.113822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] The token cache is being invalidate because project a0ccca42ac0a4113bba224c8d076dd81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:15.114053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:15.114683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5106facd-afaf-477e-aa06-7b29a9f22fae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:15.116633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1051/2110] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:14 2022] DELETE /identity/v3/projects/a0ccca42ac0a4113bba224c8d076dd81 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:15.121779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a59b5-c187-4da7-abda-fc1031b6a5ea None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.122078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a59b5-c187-4da7-abda-fc1031b6a5ea None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.122400 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a59b5-c187-4da7-abda-fc1031b6a5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.141494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ad8a59b5-c187-4da7-abda-fc1031b6a5ea None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:15.143006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ad8a59b5-c187-4da7-abda-fc1031b6a5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:15.215588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1060/2111] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:15.224936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f3357-8588-49d8-8a84-ef48acc7a7a4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.225230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f3357-8588-49d8-8a84-ef48acc7a7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.225515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f3357-8588-49d8-8a84-ef48acc7a7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.245627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d45f3357-8588-49d8-8a84-ef48acc7a7a4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:15.246935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d45f3357-8588-49d8-8a84-ef48acc7a7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:15.255314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1052/2112] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:15.259703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a59b5-c187-4da7-abda-fc1031b6a5ea None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:15.269415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a59b5-c187-4da7-abda-fc1031b6a5ea None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:15.273989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb889fd3-3d57-4ab8-b962-7f3f7aae9f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:15.274635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb889fd3-3d57-4ab8-b962-7f3f7aae9f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.274922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb889fd3-3d57-4ab8-b962-7f3f7aae9f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.275224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb889fd3-3d57-4ab8-b962-7f3f7aae9f48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.275952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb889fd3-3d57-4ab8-b962-7f3f7aae9f48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:15:15.276261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb889fd3-3d57-4ab8-b962-7f3f7aae9f48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:15.277856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb889fd3-3d57-4ab8-b962-7f3f7aae9f48 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:15.290554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1061/2113] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:15:15.294968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f3357-8588-49d8-8a84-ef48acc7a7a4 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:15.305027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f3357-8588-49d8-8a84-ef48acc7a7a4 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:15.309872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd1a6a-8904-4f66-bf99-b477a3ce7afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:15.310534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd1a6a-8904-4f66-bf99-b477a3ce7afb admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.310822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd1a6a-8904-4f66-bf99-b477a3ce7afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.311141 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd1a6a-8904-4f66-bf99-b477a3ce7afb admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.311986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd1a6a-8904-4f66-bf99-b477a3ce7afb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:15.314069 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebd1a6a-8904-4f66-bf99-b477a3ce7afb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:15.339766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1053/2114] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:15:15 2022] POST /identity/v3/projects => generated 393 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:15.343999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb889fd3-3d57-4ab8-b962-7f3f7aae9f48 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:15.356000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb889fd3-3d57-4ab8-b962-7f3f7aae9f48 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:15.417205 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9723f24-de8d-4231-bc2c-7256843091ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:15.417893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9723f24-de8d-4231-bc2c-7256843091ea admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.418196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9723f24-de8d-4231-bc2c-7256843091ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.418466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9723f24-de8d-4231-bc2c-7256843091ea admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.419227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9723f24-de8d-4231-bc2c-7256843091ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:15.421172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9723f24-de8d-4231-bc2c-7256843091ea admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:15.448873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1062/2115] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:15:15 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:15.450020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd1a6a-8904-4f66-bf99-b477a3ce7afb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:15.461752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd1a6a-8904-4f66-bf99-b477a3ce7afb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:15.467305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8459b-7e74-42bf-84fd-2a9a3d762640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:15.469170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8459b-7e74-42bf-84fd-2a9a3d762640 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.469170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8459b-7e74-42bf-84fd-2a9a3d762640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.469170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8459b-7e74-42bf-84fd-2a9a3d762640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.469170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8459b-7e74-42bf-84fd-2a9a3d762640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:15.470772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8459b-7e74-42bf-84fd-2a9a3d762640 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:15.486506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9723f24-de8d-4231-bc2c-7256843091ea admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:15.491205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1054/2116] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:15.497916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9723f24-de8d-4231-bc2c-7256843091ea admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:15.504172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086f7af-2c59-42ce-9c53-252138a3dd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:15.504845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086f7af-2c59-42ce-9c53-252138a3dd58 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.505135 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086f7af-2c59-42ce-9c53-252138a3dd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.505418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086f7af-2c59-42ce-9c53-252138a3dd58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.506208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086f7af-2c59-42ce-9c53-252138a3dd58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:15.507887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086f7af-2c59-42ce-9c53-252138a3dd58 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:15.515920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8459b-7e74-42bf-84fd-2a9a3d762640 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:15.518250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9ef8459b-7e74-42bf-84fd-2a9a3d762640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:15.518371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1063/2117] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:15.522285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086f7af-2c59-42ce-9c53-252138a3dd58 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:15.532090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086f7af-2c59-42ce-9c53-252138a3dd58 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:15.535283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8459b-7e74-42bf-84fd-2a9a3d762640 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:15.537394 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7f124b-db0f-4b42-9157-c31253d3e876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:15.538276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f124b-db0f-4b42-9157-c31253d3e876 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.538562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f124b-db0f-4b42-9157-c31253d3e876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.538845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f124b-db0f-4b42-9157-c31253d3e876 admin admin] PATH_INFO: `/v3/projects/5fd444d991ef4ed99512bedf7ef149c1/users/1514e519960e4404b9a8804a1d9c938f/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.544153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:15.545352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.546001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.546528 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] PATH_INFO: `/v3/users/c8a3657a1e3a4e40bdbb9f970cf822c4` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.548010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8a3657a1e3a4e40bdbb9f970cf822c4)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:15.550862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:15.552290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:15.556239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f124b-db0f-4b42-9157-c31253d3e876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fd444d991ef4ed99512bedf7ef149c1, user_id=1514e519960e4404b9a8804a1d9c938f, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:15.560014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7f124b-db0f-4b42-9157-c31253d3e876 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:15.589954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1064/2118] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:15:15 2022] PUT /identity/v3/projects/5fd444d991ef4ed99512bedf7ef149c1/users/1514e519960e4404b9a8804a1d9c938f/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:15:15.595460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb1c1f-4ea5-4d3d-843f-2a426ed0d32a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.595831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb1c1f-4ea5-4d3d-843f-2a426ed0d32a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.596208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb1c1f-4ea5-4d3d-843f-2a426ed0d32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.608185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8a3657a1e3a4e40bdbb9f970cf822c4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:15.609257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8a3657a1e3a4e40bdbb9f970cf822c4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:15.610011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8a3657a1e3a4e40bdbb9f970cf822c4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:15.614775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-40eb1c1f-4ea5-4d3d-843f-2a426ed0d32a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:15.616122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-40eb1c1f-4ea5-4d3d-843f-2a426ed0d32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:15.620299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8a3657a1e3a4e40bdbb9f970cf822c4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:15.629434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-575da36b-9617-44ac-9739-0ed10add45e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8a3657a1e3a4e40bdbb9f970cf822c4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:15.649607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1055/2119] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:15 2022] DELETE /identity/v3/users/c8a3657a1e3a4e40bdbb9f970cf822c4 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:15:15.654444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b860f0-bdbb-4287-a15c-16957ad0d4ca None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.654826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b860f0-bdbb-4287-a15c-16957ad0d4ca None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.655823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b860f0-bdbb-4287-a15c-16957ad0d4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.673644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-61b860f0-bdbb-4287-a15c-16957ad0d4ca None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:15.675472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-61b860f0-bdbb-4287-a15c-16957ad0d4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:15.688659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:15.707468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1065/2120] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:15:15.714915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb1c1f-4ea5-4d3d-843f-2a426ed0d32a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:15.716823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-40eb1c1f-4ea5-4d3d-843f-2a426ed0d32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:15.745809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1056/2121] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:15.755649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b860f0-bdbb-4287-a15c-16957ad0d4ca None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:15.757046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-61b860f0-bdbb-4287-a15c-16957ad0d4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:15.786716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb1c1f-4ea5-4d3d-843f-2a426ed0d32a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:15.829789 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b860f0-bdbb-4287-a15c-16957ad0d4ca None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:15.834117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bf696-0ad9-41f4-a575-f465837ad258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:15.835072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bf696-0ad9-41f4-a575-f465837ad258 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.835463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bf696-0ad9-41f4-a575-f465837ad258 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.835843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bf696-0ad9-41f4-a575-f465837ad258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.843077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dd951-9c05-4e5c-ae49-90faeaf6a7c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:15.843964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dd951-9c05-4e5c-ae49-90faeaf6a7c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:15.844388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dd951-9c05-4e5c-ae49-90faeaf6a7c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:15.844807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dd951-9c05-4e5c-ae49-90faeaf6a7c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:15.846582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bf696-0ad9-41f4-a575-f465837ad258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:15.848058 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bf696-0ad9-41f4-a575-f465837ad258 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:15.859135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1057/2122] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:15:15.864542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dd951-9c05-4e5c-ae49-90faeaf6a7c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:15.866420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dd951-9c05-4e5c-ae49-90faeaf6a7c3 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:15.879249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1066/2123] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:15:16.290677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bf696-0ad9-41f4-a575-f465837ad258 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:16.293205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-983bf696-0ad9-41f4-a575-f465837ad258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:16.320393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bf696-0ad9-41f4-a575-f465837ad258 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:16.325149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:16.325801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:16.326092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:16.326368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] PATH_INFO: `/v3/projects/412f781a4ac24d06ab713b8a63383d5a` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:16.335071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=412f781a4ac24d06ab713b8a63383d5a)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:16.337094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:16.410983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412f781a4ac24d06ab713b8a63383d5a', 'request_id': 'req-4d10b38f-17a0-4554-a92a-7037644aace9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:16.421618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412f781a4ac24d06ab713b8a63383d5a', 'request_id': 'req-4d10b38f-17a0-4554-a92a-7037644aace9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:16.440048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412f781a4ac24d06ab713b8a63383d5a', 'request_id': 'req-4d10b38f-17a0-4554-a92a-7037644aace9'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:16.458308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] The token cache is being invalidate because project 412f781a4ac24d06ab713b8a63383d5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:16.459751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:16.460563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:16.466841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1058/2124] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:16 2022] DELETE /identity/v3/projects/412f781a4ac24d06ab713b8a63383d5a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:16.468201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff58b02-87a8-4a5e-9016-f5cc66de588a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:16.468653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff58b02-87a8-4a5e-9016-f5cc66de588a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:16.468984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff58b02-87a8-4a5e-9016-f5cc66de588a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:16.495135 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-3ff58b02-87a8-4a5e-9016-f5cc66de588a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:16.496700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3ff58b02-87a8-4a5e-9016-f5cc66de588a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:16.541894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:16.541894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:16.561005 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:16.586912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1067/2125] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:15:16.595684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba23395-1d83-4d9a-ac8c-764044a08428 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:16.596071 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba23395-1d83-4d9a-ac8c-764044a08428 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:16.596854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba23395-1d83-4d9a-ac8c-764044a08428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:16.617677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:16.637230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-7ba23395-1d83-4d9a-ac8c-764044a08428 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:16.638374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7ba23395-1d83-4d9a-ac8c-764044a08428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:16.648700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10b38f-17a0-4554-a92a-7037644aace9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:16.652768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1068/2126] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:16.656742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba23395-1d83-4d9a-ac8c-764044a08428 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:16.657982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:16.659585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:16.659970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:16.660348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] PATH_INFO: `/v3/users/b2ecddfd823049659ebdd10facf493ed` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:16.661684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2ecddfd823049659ebdd10facf493ed)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:16.664035 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:16.672616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba23395-1d83-4d9a-ac8c-764044a08428 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:16.682719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f6384-719f-4fc4-8613-be0068d53952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:16.683916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f6384-719f-4fc4-8613-be0068d53952 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:16.686878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f6384-719f-4fc4-8613-be0068d53952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:16.687443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f6384-719f-4fc4-8613-be0068d53952 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:16.688995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f6384-719f-4fc4-8613-be0068d53952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:15:16.689303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f6384-719f-4fc4-8613-be0068d53952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:16.695332 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f6384-719f-4fc4-8613-be0068d53952 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:16.717303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1069/2127] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:15:16.722045 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f6384-719f-4fc4-8613-be0068d53952 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:16.736938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f6384-719f-4fc4-8613-be0068d53952 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:16.744160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e78bc0-39c6-4cd7-ad49-d63a35cba54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:16.745052 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e78bc0-39c6-4cd7-ad49-d63a35cba54c admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:16.745523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e78bc0-39c6-4cd7-ad49-d63a35cba54c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:16.746255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e78bc0-39c6-4cd7-ad49-d63a35cba54c admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:16.746845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e78bc0-39c6-4cd7-ad49-d63a35cba54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:16.749979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e78bc0-39c6-4cd7-ad49-d63a35cba54c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:16.759294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2ecddfd823049659ebdd10facf493ed'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:16.760517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2ecddfd823049659ebdd10facf493ed'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:16.761398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2ecddfd823049659ebdd10facf493ed'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:16.772714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2ecddfd823049659ebdd10facf493ed'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:16.781291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-783c1277-7d7f-4b61-bfd7-dadb05a9e06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2ecddfd823049659ebdd10facf493ed'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:16.810813 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1070/2128] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:15:16 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:16.815410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e78bc0-39c6-4cd7-ad49-d63a35cba54c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:16.817231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-57e78bc0-39c6-4cd7-ad49-d63a35cba54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:16.817743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1059/2129] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:16 2022] DELETE /identity/v3/users/b2ecddfd823049659ebdd10facf493ed => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:15:16.822002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7cd87-9901-45ce-ae12-91571043bef2 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:16.822299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7cd87-9901-45ce-ae12-91571043bef2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:16.822581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e7cd87-9901-45ce-ae12-91571043bef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:16.844005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-27e7cd87-9901-45ce-ae12-91571043bef2 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:16.845581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-27e7cd87-9901-45ce-ae12-91571043bef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:16.907721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e78bc0-39c6-4cd7-ad49-d63a35cba54c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:16.941226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1060/2130] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:15:16.951115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7cd87-9901-45ce-ae12-91571043bef2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:16.953385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-27e7cd87-9901-45ce-ae12-91571043bef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:16.990575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeee10c-d32d-4ac8-9a3a-f8551b4ca4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:16.991658 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeee10c-d32d-4ac8-9a3a-f8551b4ca4ce admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:16.992051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeee10c-d32d-4ac8-9a3a-f8551b4ca4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:16.992430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeee10c-d32d-4ac8-9a3a-f8551b4ca4ce admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:16.993791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeee10c-d32d-4ac8-9a3a-f8551b4ca4ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:16.996487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeee10c-d32d-4ac8-9a3a-f8551b4ca4ce admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:17.027682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1071/2131] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:15:16 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:17.035769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeee10c-d32d-4ac8-9a3a-f8551b4ca4ce admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:17.037628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e7cd87-9901-45ce-ae12-91571043bef2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:17.042145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6ce52-f614-4f3c-96a5-5d50ce2750dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:17.042809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6ce52-f614-4f3c-96a5-5d50ce2750dd service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:17.043156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6ce52-f614-4f3c-96a5-5d50ce2750dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:17.043443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d6ce52-f614-4f3c-96a5-5d50ce2750dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:17.051512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeee10c-d32d-4ac8-9a3a-f8551b4ca4ce admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:17.054982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6ce52-f614-4f3c-96a5-5d50ce2750dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:17.056548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d6ce52-f614-4f3c-96a5-5d50ce2750dd service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:17.057099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29144084-a322-4d32-b5e9-051707e67720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:17.057843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29144084-a322-4d32-b5e9-051707e67720 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:17.058198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29144084-a322-4d32-b5e9-051707e67720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:17.058619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29144084-a322-4d32-b5e9-051707e67720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:17.059448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29144084-a322-4d32-b5e9-051707e67720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:17.061273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29144084-a322-4d32-b5e9-051707e67720 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:17.067493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1061/2132] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:17.080971 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1072/2133] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:17.082582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6ce52-f614-4f3c-96a5-5d50ce2750dd service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:17.093860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d6ce52-f614-4f3c-96a5-5d50ce2750dd service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:17.101551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8be00-c898-468d-93c9-9147c9a22cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:17.102258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8be00-c898-468d-93c9-9147c9a22cad admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:17.102546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8be00-c898-468d-93c9-9147c9a22cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:17.105269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8be00-c898-468d-93c9-9147c9a22cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:17.106328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8be00-c898-468d-93c9-9147c9a22cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:17.109310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8be00-c898-468d-93c9-9147c9a22cad admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:17.114614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:17.129078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1062/2134] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:15:17.130837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29144084-a322-4d32-b5e9-051707e67720 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:17.147113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29144084-a322-4d32-b5e9-051707e67720 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:17.161478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cef34b-5e70-489a-843e-c702bb2d3b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:17.162501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cef34b-5e70-489a-843e-c702bb2d3b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:17.162786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cef34b-5e70-489a-843e-c702bb2d3b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:17.163097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cef34b-5e70-489a-843e-c702bb2d3b57 admin admin] PATH_INFO: `/v3/projects/8f9bcf35ce214499b641a715a7f53b60/users/50bbf05113164b7cb42bebaa75abf75e/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:17.188355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cef34b-5e70-489a-843e-c702bb2d3b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9bcf35ce214499b641a715a7f53b60, user_id=50bbf05113164b7cb42bebaa75abf75e, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:17.207928 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cef34b-5e70-489a-843e-c702bb2d3b57 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:17.229073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1073/2135] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:15:17 2022] PUT /identity/v3/projects/8f9bcf35ce214499b641a715a7f53b60/users/50bbf05113164b7cb42bebaa75abf75e/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:17.299476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc272404-212d-4fc7-9f04-723424ec6311 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:17.299653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc272404-212d-4fc7-9f04-723424ec6311 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:17.300814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc272404-212d-4fc7-9f04-723424ec6311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:17.363568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fc272404-212d-4fc7-9f04-723424ec6311 None None] MFA Rules not processed for user `50bbf05113164b7cb42bebaa75abf75e`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:17.366404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fc272404-212d-4fc7-9f04-723424ec6311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:17.471605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cef34b-5e70-489a-843e-c702bb2d3b57 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:17.474626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-02cef34b-5e70-489a-843e-c702bb2d3b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:17.475556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1063/2136] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:17.497707 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:17.498723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc272404-212d-4fc7-9f04-723424ec6311 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:17.500930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fc272404-212d-4fc7-9f04-723424ec6311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:17.615538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cef34b-5e70-489a-843e-c702bb2d3b57 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:17.630996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:17.631933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:17.632213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:17.632490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] PATH_INFO: `/v3/projects/8ca5af1b59244f798024cbda56f7e8fc` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:17.634240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ca5af1b59244f798024cbda56f7e8fc)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:17.636267 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:17.650622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc272404-212d-4fc7-9f04-723424ec6311 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:17.657991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef0999-c12f-466b-bff1-47745431f836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:17.661327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef0999-c12f-466b-bff1-47745431f836 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:17.661622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef0999-c12f-466b-bff1-47745431f836 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:17.661904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef0999-c12f-466b-bff1-47745431f836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:17.677277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef0999-c12f-466b-bff1-47745431f836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:17.678746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ef0999-c12f-466b-bff1-47745431f836 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:17.693969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1064/2137] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:17.725800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef0999-c12f-466b-bff1-47745431f836 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:17.742884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef0999-c12f-466b-bff1-47745431f836 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:17.747615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ca5af1b59244f798024cbda56f7e8fc', 'request_id': 'req-6668fd1c-3c49-4abe-a2de-e657ede948f6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:17.758542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ca5af1b59244f798024cbda56f7e8fc', 'request_id': 'req-6668fd1c-3c49-4abe-a2de-e657ede948f6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:17.766385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ca5af1b59244f798024cbda56f7e8fc', 'request_id': 'req-6668fd1c-3c49-4abe-a2de-e657ede948f6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:17.782598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] The token cache is being invalidate because project 8ca5af1b59244f798024cbda56f7e8fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:17.783212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:17.784325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-6668fd1c-3c49-4abe-a2de-e657ede948f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:17.787007 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1074/2138] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:17 2022] DELETE /identity/v3/projects/8ca5af1b59244f798024cbda56f7e8fc => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:17.791545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528659f0-65e5-41ef-a917-50f666110c17 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:17.791962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528659f0-65e5-41ef-a917-50f666110c17 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:17.792412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528659f0-65e5-41ef-a917-50f666110c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:17.824575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-528659f0-65e5-41ef-a917-50f666110c17 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:17.825722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-528659f0-65e5-41ef-a917-50f666110c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:17.839656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:17.840592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:17.841054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:17.841493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] PATH_INFO: `/v3/users/50bbf05113164b7cb42bebaa75abf75e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:17.843413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50bbf05113164b7cb42bebaa75abf75e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:17.846064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:17.926998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:17.944809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1075/2139] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:15:17.951788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50bbf05113164b7cb42bebaa75abf75e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:17.952624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50bbf05113164b7cb42bebaa75abf75e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:17.952991 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f247-52b5-4ccf-9998-5a72d7f52e64 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:17.953780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f247-52b5-4ccf-9998-5a72d7f52e64 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:17.953889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50bbf05113164b7cb42bebaa75abf75e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:17.954124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f247-52b5-4ccf-9998-5a72d7f52e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:17.966317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50bbf05113164b7cb42bebaa75abf75e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:17.976563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-49603eef-a59d-457d-8f98-5089a4e14ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50bbf05113164b7cb42bebaa75abf75e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:17.983741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5cf7f247-52b5-4ccf-9998-5a72d7f52e64 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:17.986144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5cf7f247-52b5-4ccf-9998-5a72d7f52e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:17.997875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1076/2140] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:18.000900 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7f247-52b5-4ccf-9998-5a72d7f52e64 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:18.011348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1065/2141] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:17 2022] DELETE /identity/v3/users/50bbf05113164b7cb42bebaa75abf75e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:15:18.013189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7f247-52b5-4ccf-9998-5a72d7f52e64 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:18.016045 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30738-1f0e-4654-8efc-df16b4471d93 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.016334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30738-1f0e-4654-8efc-df16b4471d93 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.016616 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae30738-1f0e-4654-8efc-df16b4471d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.041034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9ae30738-1f0e-4654-8efc-df16b4471d93 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:18.042542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9ae30738-1f0e-4654-8efc-df16b4471d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:18.056717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:18.092250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48933549-4b07-40d7-86f8-772c0fca2c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:18.093038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48933549-4b07-40d7-86f8-772c0fca2c44 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.093351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48933549-4b07-40d7-86f8-772c0fca2c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.093784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48933549-4b07-40d7-86f8-772c0fca2c44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.095592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48933549-4b07-40d7-86f8-772c0fca2c44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:15:18.095715 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48933549-4b07-40d7-86f8-772c0fca2c44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:18.099086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48933549-4b07-40d7-86f8-772c0fca2c44 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:18.113407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1077/2142] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:15:18.127380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48933549-4b07-40d7-86f8-772c0fca2c44 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:18.130116 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1066/2143] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:15:18.149193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae30738-1f0e-4654-8efc-df16b4471d93 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:18.151146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9ae30738-1f0e-4654-8efc-df16b4471d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:18.153463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48933549-4b07-40d7-86f8-772c0fca2c44 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:18.164604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de8b8f-6575-4bde-918c-1adde549ec59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:18.165641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de8b8f-6575-4bde-918c-1adde549ec59 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.166132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de8b8f-6575-4bde-918c-1adde549ec59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.166622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de8b8f-6575-4bde-918c-1adde549ec59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.167723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de8b8f-6575-4bde-918c-1adde549ec59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:18.169898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de8b8f-6575-4bde-918c-1adde549ec59 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:18.222289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1078/2144] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:15:18 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:18.226212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de8b8f-6575-4bde-918c-1adde549ec59 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:18.243233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae30738-1f0e-4654-8efc-df16b4471d93 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:18.325659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de8b8f-6575-4bde-918c-1adde549ec59 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:18.331329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec605b11-bb0e-4818-9802-4b234a0a4771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:18.332015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec605b11-bb0e-4818-9802-4b234a0a4771 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.332300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec605b11-bb0e-4818-9802-4b234a0a4771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.332575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec605b11-bb0e-4818-9802-4b234a0a4771 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.333405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec605b11-bb0e-4818-9802-4b234a0a4771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:18.334473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b071c49-fe0d-4ed0-9622-73bc16e6a8eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:18.335551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec605b11-bb0e-4818-9802-4b234a0a4771 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:18.335682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b071c49-fe0d-4ed0-9622-73bc16e6a8eb service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.335982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b071c49-fe0d-4ed0-9622-73bc16e6a8eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.336405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b071c49-fe0d-4ed0-9622-73bc16e6a8eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.351587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b071c49-fe0d-4ed0-9622-73bc16e6a8eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:18.354046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b071c49-fe0d-4ed0-9622-73bc16e6a8eb service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:18.369656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1067/2145] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:15:18.371534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1079/2146] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:15:18 2022] POST /identity/v3/users => generated 285 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:18.375472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b071c49-fe0d-4ed0-9622-73bc16e6a8eb service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:18.389977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b071c49-fe0d-4ed0-9622-73bc16e6a8eb service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:18.395289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd59462-c84e-40ee-b8ad-e3af0afd4126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:18.395852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd59462-c84e-40ee-b8ad-e3af0afd4126 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.396138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd59462-c84e-40ee-b8ad-e3af0afd4126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.396656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd59462-c84e-40ee-b8ad-e3af0afd4126 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.397152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd59462-c84e-40ee-b8ad-e3af0afd4126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:18.398889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd59462-c84e-40ee-b8ad-e3af0afd4126 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:18.401597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:18.412918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1068/2147] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:15:18.416184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec605b11-bb0e-4818-9802-4b234a0a4771 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:18.429431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec605b11-bb0e-4818-9802-4b234a0a4771 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:18.436892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefdab2-6204-41e1-844f-82e1b8ef0df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:18.437826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefdab2-6204-41e1-844f-82e1b8ef0df1 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.438624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefdab2-6204-41e1-844f-82e1b8ef0df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.438624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefdab2-6204-41e1-844f-82e1b8ef0df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.439423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefdab2-6204-41e1-844f-82e1b8ef0df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:18.441169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefdab2-6204-41e1-844f-82e1b8ef0df1 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:18.458187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1080/2148] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:18.459392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd59462-c84e-40ee-b8ad-e3af0afd4126 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:18.472509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd59462-c84e-40ee-b8ad-e3af0afd4126 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:18.478613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0797da-ce73-4c49-98d9-1a4477392c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:18.479556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0797da-ce73-4c49-98d9-1a4477392c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.479894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0797da-ce73-4c49-98d9-1a4477392c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.480180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0797da-ce73-4c49-98d9-1a4477392c05 admin admin] PATH_INFO: `/v3/projects/5a5cb1b4434441a89f7aa54b3fae5625/users/0b233edb0fd04c7f978aab9eddaf4020/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.499945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0797da-ce73-4c49-98d9-1a4477392c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5cb1b4434441a89f7aa54b3fae5625, user_id=0b233edb0fd04c7f978aab9eddaf4020, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:18.504821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0797da-ce73-4c49-98d9-1a4477392c05 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:18.522411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1069/2149] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:15:18 2022] PUT /identity/v3/projects/5a5cb1b4434441a89f7aa54b3fae5625/users/0b233edb0fd04c7f978aab9eddaf4020/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:15:18.532329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1ee94-8c45-42f2-83ed-ff39d33d3092 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.532855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1ee94-8c45-42f2-83ed-ff39d33d3092 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.532937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f1ee94-8c45-42f2-83ed-ff39d33d3092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.574871 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-40f1ee94-8c45-42f2-83ed-ff39d33d3092 None None] MFA Rules not processed for user `0b233edb0fd04c7f978aab9eddaf4020`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:18.576502 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-40f1ee94-8c45-42f2-83ed-ff39d33d3092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:18.674808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0797da-ce73-4c49-98d9-1a4477392c05 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:18.676648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8e0797da-ce73-4c49-98d9-1a4477392c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:18.680585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1081/2150] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:18.692024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1ee94-8c45-42f2-83ed-ff39d33d3092 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:18.694001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-40f1ee94-8c45-42f2-83ed-ff39d33d3092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:18.726950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:18.784865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f1ee94-8c45-42f2-83ed-ff39d33d3092 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:18.789304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0797da-ce73-4c49-98d9-1a4477392c05 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:18.791814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755cc2f-59fb-421c-aede-72ec307d0cae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450bcd0f22b840c299d08ce9433d6c85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:18.793017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755cc2f-59fb-421c-aede-72ec307d0cae service glance] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.793409 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755cc2f-59fb-421c-aede-72ec307d0cae service glance] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.794745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755cc2f-59fb-421c-aede-72ec307d0cae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.798143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:18.798931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:18.801421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:18.801669 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] PATH_INFO: `/v3/projects/8f9bcf35ce214499b641a715a7f53b60` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:18.803359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9bcf35ce214499b641a715a7f53b60)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:18.810883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:18.814979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755cc2f-59fb-421c-aede-72ec307d0cae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:18.816621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755cc2f-59fb-421c-aede-72ec307d0cae service glance] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:18.838435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1082/2151] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:15:18.963421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9bcf35ce214499b641a715a7f53b60', 'request_id': 'req-01c58939-09aa-44d0-9a61-0d3a1d02681d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:18.977256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9bcf35ce214499b641a715a7f53b60', 'request_id': 'req-01c58939-09aa-44d0-9a61-0d3a1d02681d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:18.987303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9bcf35ce214499b641a715a7f53b60', 'request_id': 'req-01c58939-09aa-44d0-9a61-0d3a1d02681d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:19.007389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] The token cache is being invalidate because project 8f9bcf35ce214499b641a715a7f53b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:19.007733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:19.008853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-01c58939-09aa-44d0-9a61-0d3a1d02681d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:19.013224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1070/2152] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:18 2022] DELETE /identity/v3/projects/8f9bcf35ce214499b641a715a7f53b60 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:20.433448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755cc2f-59fb-421c-aede-72ec307d0cae service glance] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:20.435648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2755cc2f-59fb-421c-aede-72ec307d0cae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:20.454147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755cc2f-59fb-421c-aede-72ec307d0cae service glance] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:20.550672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:20.552749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:20.552984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:20.553276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] PATH_INFO: `/v3/users/bafc322cedbe4f3aa24ff8567a2767d4` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:20.554921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bafc322cedbe4f3aa24ff8567a2767d4)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:20.557292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:20.629390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bafc322cedbe4f3aa24ff8567a2767d4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:20.630148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bafc322cedbe4f3aa24ff8567a2767d4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:20.630767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bafc322cedbe4f3aa24ff8567a2767d4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:20.639441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bafc322cedbe4f3aa24ff8567a2767d4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:20.648141 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bafc322cedbe4f3aa24ff8567a2767d4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:20.668491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1083/2153] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:20 2022] DELETE /identity/v3/users/bafc322cedbe4f3aa24ff8567a2767d4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:20.673022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6f15-a84b-47b6-829b-413e6d2486a9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:20.673550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6f15-a84b-47b6-829b-413e6d2486a9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:20.673631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d6f15-a84b-47b6-829b-413e6d2486a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:20.693519 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f63d6f15-a84b-47b6-829b-413e6d2486a9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:20.695152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f63d6f15-a84b-47b6-829b-413e6d2486a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:20.770613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1071/2154] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:20.782374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:20.784666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:20.857884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:20.880957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cef83-6ada-497e-928c-cbc7b5a13c8d admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:20.886893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3713b-33dd-4931-b37f-cb6d4465eacf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:20.886893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3713b-33dd-4931-b37f-cb6d4465eacf service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:20.886893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3713b-33dd-4931-b37f-cb6d4465eacf service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:20.886893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3713b-33dd-4931-b37f-cb6d4465eacf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:20.898137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3713b-33dd-4931-b37f-cb6d4465eacf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:20.900028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3713b-33dd-4931-b37f-cb6d4465eacf service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:20.914112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1084/2155] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:21.312764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8be81-e693-47d2-94b5-8730de06fe90 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:21.314183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8be81-e693-47d2-94b5-8730de06fe90 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:21.314474 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8be81-e693-47d2-94b5-8730de06fe90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:21.324465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:21.355108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-87f8be81-e693-47d2-94b5-8730de06fe90 None None] MFA Rules not processed for user `1514e519960e4404b9a8804a1d9c938f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:21.356445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-87f8be81-e693-47d2-94b5-8730de06fe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:21.484988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1072/2156] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:21.497091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae51ec3-aebe-4848-868a-7c1ab9370268 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:21.497634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae51ec3-aebe-4848-868a-7c1ab9370268 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:21.497634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae51ec3-aebe-4848-868a-7c1ab9370268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:21.524552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0ae51ec3-aebe-4848-868a-7c1ab9370268 None None] MFA Rules not processed for user `1514e519960e4404b9a8804a1d9c938f`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:21.525901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0ae51ec3-aebe-4848-868a-7c1ab9370268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:21.547143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1085/2157] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:21.556315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8be81-e693-47d2-94b5-8730de06fe90 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:21.557971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-87f8be81-e693-47d2-94b5-8730de06fe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:21.596448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae51ec3-aebe-4848-868a-7c1ab9370268 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:21.598418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0ae51ec3-aebe-4848-868a-7c1ab9370268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:21.637835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae51ec3-aebe-4848-868a-7c1ab9370268 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:21.660628 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:21.661526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:21.661814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:21.662090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] PATH_INFO: `/v3/projects/0ff25c5cca1d44179e2247b8cfb00aff` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:21.663352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ff25c5cca1d44179e2247b8cfb00aff)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:21.665306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:21.693146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8be81-e693-47d2-94b5-8730de06fe90 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:21.710683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e44bce-6ba0-4f18-942c-cf184e5dabeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:21.712028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e44bce-6ba0-4f18-942c-cf184e5dabeb service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:21.712875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e44bce-6ba0-4f18-942c-cf184e5dabeb service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:21.713235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e44bce-6ba0-4f18-942c-cf184e5dabeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:21.731421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e44bce-6ba0-4f18-942c-cf184e5dabeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:21.736325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e44bce-6ba0-4f18-942c-cf184e5dabeb service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:21.746515 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ff25c5cca1d44179e2247b8cfb00aff', 'request_id': 'req-03ede48c-5087-48df-a3bc-e6f3a3e6be98'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:21.758148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ff25c5cca1d44179e2247b8cfb00aff', 'request_id': 'req-03ede48c-5087-48df-a3bc-e6f3a3e6be98'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:21.761867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1073/2158] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:15:21.800822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ff25c5cca1d44179e2247b8cfb00aff', 'request_id': 'req-03ede48c-5087-48df-a3bc-e6f3a3e6be98'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:21.814390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] The token cache is being invalidate because project 0ff25c5cca1d44179e2247b8cfb00aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:21.815477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:21.816287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-03ede48c-5087-48df-a3bc-e6f3a3e6be98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:21.818259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1086/2159] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:21 2022] DELETE /identity/v3/projects/0ff25c5cca1d44179e2247b8cfb00aff => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:21.823344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30628f-1e45-4c2e-9bc5-be7a5af775b4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:21.823519 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30628f-1e45-4c2e-9bc5-be7a5af775b4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:21.823778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30628f-1e45-4c2e-9bc5-be7a5af775b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:21.878721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fd30628f-1e45-4c2e-9bc5-be7a5af775b4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:21.880689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fd30628f-1e45-4c2e-9bc5-be7a5af775b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:22.056055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1074/2160] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:15:22.064126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e0039-2668-477d-bcdd-684eec9cfba9 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:22.064403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e0039-2668-477d-bcdd-684eec9cfba9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:22.065226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e0039-2668-477d-bcdd-684eec9cfba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:22.095617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-6b5e0039-2668-477d-bcdd-684eec9cfba9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:22.097180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6b5e0039-2668-477d-bcdd-684eec9cfba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:22.108269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1087/2161] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:22.112172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30628f-1e45-4c2e-9bc5-be7a5af775b4 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:22.132391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30628f-1e45-4c2e-9bc5-be7a5af775b4 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:22.137998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a3e78-13f8-44d9-9914-64dce73ec8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:22.139145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a3e78-13f8-44d9-9914-64dce73ec8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:22.139646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a3e78-13f8-44d9-9914-64dce73ec8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:22.140135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a3e78-13f8-44d9-9914-64dce73ec8c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:22.141388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a3e78-13f8-44d9-9914-64dce73ec8c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:15:22.143383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a3e78-13f8-44d9-9914-64dce73ec8c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:22.145683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a3e78-13f8-44d9-9914-64dce73ec8c3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:22.171610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1075/2162] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:15:22.176116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e0039-2668-477d-bcdd-684eec9cfba9 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:22.196417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e0039-2668-477d-bcdd-684eec9cfba9 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:22.207573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ce6b-2618-4b0d-84b2-6395a0045cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:22.208266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ce6b-2618-4b0d-84b2-6395a0045cdd admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:22.208563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ce6b-2618-4b0d-84b2-6395a0045cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:22.208846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526ce6b-2618-4b0d-84b2-6395a0045cdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:22.209561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526ce6b-2618-4b0d-84b2-6395a0045cdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:22.211748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526ce6b-2618-4b0d-84b2-6395a0045cdd admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:22.253915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1088/2163] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:15:22 2022] POST /identity/v3/projects => generated 391 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:22.258779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a3e78-13f8-44d9-9914-64dce73ec8c3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:22.281106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a3e78-13f8-44d9-9914-64dce73ec8c3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:22.295480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:22.385251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12281115-afb0-4634-9632-bc84619d446d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:22.385962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12281115-afb0-4634-9632-bc84619d446d admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:22.386258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12281115-afb0-4634-9632-bc84619d446d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:22.386546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12281115-afb0-4634-9632-bc84619d446d admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:22.388184 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12281115-afb0-4634-9632-bc84619d446d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:22.390545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12281115-afb0-4634-9632-bc84619d446d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:22.435000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1076/2164] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:15:22 2022] POST /identity/v3/users => generated 296 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:22.435793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ce6b-2618-4b0d-84b2-6395a0045cdd admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:22.457906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526ce6b-2618-4b0d-84b2-6395a0045cdd admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:22.467096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2133f3c-72d3-4d7c-ae72-1bd08dd239d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:22.467782 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2133f3c-72d3-4d7c-ae72-1bd08dd239d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:22.468099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2133f3c-72d3-4d7c-ae72-1bd08dd239d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:22.468390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2133f3c-72d3-4d7c-ae72-1bd08dd239d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:22.469151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2133f3c-72d3-4d7c-ae72-1bd08dd239d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:22.477781 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2133f3c-72d3-4d7c-ae72-1bd08dd239d7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:22.496117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1089/2165] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:22.499866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12281115-afb0-4634-9632-bc84619d446d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:22.512624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12281115-afb0-4634-9632-bc84619d446d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:22.520620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a561d7c-1c28-4230-99ce-7a270aacacfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:22.521502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a561d7c-1c28-4230-99ce-7a270aacacfe admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:22.521803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a561d7c-1c28-4230-99ce-7a270aacacfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:22.522088 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a561d7c-1c28-4230-99ce-7a270aacacfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:22.524584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a561d7c-1c28-4230-99ce-7a270aacacfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:22.526587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a561d7c-1c28-4230-99ce-7a270aacacfe admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:22.543996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1077/2166] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:22.545548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2133f3c-72d3-4d7c-ae72-1bd08dd239d7 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:22.560327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2133f3c-72d3-4d7c-ae72-1bd08dd239d7 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:22.567996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8ef55-5018-4376-b89b-34303fdcf909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:22.569290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8ef55-5018-4376-b89b-34303fdcf909 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:22.569791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8ef55-5018-4376-b89b-34303fdcf909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:22.570297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8ef55-5018-4376-b89b-34303fdcf909 admin admin] PATH_INFO: `/v3/projects/664ce4bffd5b44ca827878e6a24ab3f5/users/b23cf24e2c6748cbacece238b03b6467/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:22.592929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8ef55-5018-4376-b89b-34303fdcf909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=664ce4bffd5b44ca827878e6a24ab3f5, user_id=b23cf24e2c6748cbacece238b03b6467, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:22.595882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8ef55-5018-4376-b89b-34303fdcf909 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:22.613978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1090/2167] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:15:22 2022] PUT /identity/v3/projects/664ce4bffd5b44ca827878e6a24ab3f5/users/b23cf24e2c6748cbacece238b03b6467/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:22.625384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcff430-a3a7-4cb2-b059-885d6750c50e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:22.625760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcff430-a3a7-4cb2-b059-885d6750c50e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:22.626145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcff430-a3a7-4cb2-b059-885d6750c50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:22.666502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-6fcff430-a3a7-4cb2-b059-885d6750c50e None None] MFA Rules not processed for user `b23cf24e2c6748cbacece238b03b6467`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:22.667900 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6fcff430-a3a7-4cb2-b059-885d6750c50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:22.762355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1078/2168] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:22.773804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8ef55-5018-4376-b89b-34303fdcf909 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:22.775684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b8d8ef55-5018-4376-b89b-34303fdcf909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:22.869029 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8ef55-5018-4376-b89b-34303fdcf909 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:22.873054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e802018-67f2-41c6-916e-8682c407a31c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:22.873971 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e802018-67f2-41c6-916e-8682c407a31c service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:22.874117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e802018-67f2-41c6-916e-8682c407a31c service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:22.874476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e802018-67f2-41c6-916e-8682c407a31c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:22.884617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:22.887862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e802018-67f2-41c6-916e-8682c407a31c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:22.889974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e802018-67f2-41c6-916e-8682c407a31c service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:22.918946 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1091/2169] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:23.076294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c9162-d797-41d0-acc0-7a1eef3bc803 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:23.076691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c9162-d797-41d0-acc0-7a1eef3bc803 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:23.077118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c9162-d797-41d0-acc0-7a1eef3bc803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:23.116184 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-dd7c9162-d797-41d0-acc0-7a1eef3bc803 None None] MFA Rules not processed for user `1514e519960e4404b9a8804a1d9c938f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:23.117420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-dd7c9162-d797-41d0-acc0-7a1eef3bc803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:23.217773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1079/2170] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:23.362200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e802018-67f2-41c6-916e-8682c407a31c service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:23.364865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2e802018-67f2-41c6-916e-8682c407a31c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:23.527912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e802018-67f2-41c6-916e-8682c407a31c service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:23.537747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:23.538678 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:23.539182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:23.539744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] PATH_INFO: `/v3/users/dd3cd89d5f4f468bbb35d74f810dec27` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:23.541626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd3cd89d5f4f468bbb35d74f810dec27)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:23.544471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:23.624466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd3cd89d5f4f468bbb35d74f810dec27'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:23.628580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd3cd89d5f4f468bbb35d74f810dec27'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:23.628580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd3cd89d5f4f468bbb35d74f810dec27'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:23.662390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd3cd89d5f4f468bbb35d74f810dec27'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:23.673537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd3cd89d5f4f468bbb35d74f810dec27'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:23.706374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1092/2171] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:23 2022] DELETE /identity/v3/users/dd3cd89d5f4f468bbb35d74f810dec27 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:23.935275 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c9162-d797-41d0-acc0-7a1eef3bc803 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:23.952481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c9162-d797-41d0-acc0-7a1eef3bc803 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:24.019375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:24.082018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:24.082991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:24.083452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:24.083805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] PATH_INFO: `/v3/projects/16a06122e53d44faa473f56f09d70ff3` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:24.085202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16a06122e53d44faa473f56f09d70ff3)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:24.089038 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:24.145664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16a06122e53d44faa473f56f09d70ff3', 'request_id': 'req-1b0b420d-4732-45bf-b4c0-a20805ddfb05'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:24.155645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16a06122e53d44faa473f56f09d70ff3', 'request_id': 'req-1b0b420d-4732-45bf-b4c0-a20805ddfb05'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:24.164107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16a06122e53d44faa473f56f09d70ff3', 'request_id': 'req-1b0b420d-4732-45bf-b4c0-a20805ddfb05'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:24.179436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] The token cache is being invalidate because project 16a06122e53d44faa473f56f09d70ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:24.180500 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:24.181574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:24.184388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1080/2172] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:23 2022] DELETE /identity/v3/projects/16a06122e53d44faa473f56f09d70ff3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:24.190987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:24.193503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:24.208465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739c5bd8-ee3d-4bbd-8e8f-f2b38be9329a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:24.312643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:24.313636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:24.314827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:24.314827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] PATH_INFO: `/v3/users/1d9ab91875cd49e3a10ac52dbd35b2f0` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:24.316283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9ab91875cd49e3a10ac52dbd35b2f0)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:24.318871 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:24.347784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:24.432516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9ab91875cd49e3a10ac52dbd35b2f0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:24.433809 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9ab91875cd49e3a10ac52dbd35b2f0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:24.434874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9ab91875cd49e3a10ac52dbd35b2f0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:24.449800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9ab91875cd49e3a10ac52dbd35b2f0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:24.461896 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-b314c6da-79f9-4f1a-a7ea-3177c71e6105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9ab91875cd49e3a10ac52dbd35b2f0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:24.538995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1093/2173] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:24 2022] DELETE /identity/v3/users/1d9ab91875cd49e3a10ac52dbd35b2f0 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:24.858830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:24.861006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:24.889576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b420d-4732-45bf-b4c0-a20805ddfb05 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:25.028979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:25.030901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:25.031164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:25.032977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] PATH_INFO: `/v3/projects/44792fb74fe345d19680430a2f3a909f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:25.036695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] RBAC: Authorizing `identity:delete_project(project_id=44792fb74fe345d19680430a2f3a909f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:25.038741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:25.155275 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44792fb74fe345d19680430a2f3a909f', 'request_id': 'req-3aee45b8-285b-491a-8ecb-a336161404be'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:25.159603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:25.169729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44792fb74fe345d19680430a2f3a909f', 'request_id': 'req-3aee45b8-285b-491a-8ecb-a336161404be'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:25.182861 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44792fb74fe345d19680430a2f3a909f', 'request_id': 'req-3aee45b8-285b-491a-8ecb-a336161404be'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:25.202897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] The token cache is being invalidate because project 44792fb74fe345d19680430a2f3a909f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:25.203374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:25.204545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3aee45b8-285b-491a-8ecb-a336161404be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:25.208960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1081/2174] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:24 2022] DELETE /identity/v3/projects/44792fb74fe345d19680430a2f3a909f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:25.214010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a550338-32da-4826-ac26-21ea00214dc5 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:25.214491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a550338-32da-4826-ac26-21ea00214dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:25.214608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a550338-32da-4826-ac26-21ea00214dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:25.233571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4a550338-32da-4826-ac26-21ea00214dc5 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:25.234848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4a550338-32da-4826-ac26-21ea00214dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:25.312292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1094/2175] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:25.320858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab87fc-330d-40d9-bd66-012ff9b66311 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:25.321152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab87fc-330d-40d9-bd66-012ff9b66311 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:25.321516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab87fc-330d-40d9-bd66-012ff9b66311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:25.348309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2fab87fc-330d-40d9-bd66-012ff9b66311 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:25.349737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2fab87fc-330d-40d9-bd66-012ff9b66311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:25.368641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1082/2176] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:25.373435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a550338-32da-4826-ac26-21ea00214dc5 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:25.405104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a550338-32da-4826-ac26-21ea00214dc5 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:25.416389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5db68e-d499-4ae5-9e19-d07d2f7a7b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:25.417082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5db68e-d499-4ae5-9e19-d07d2f7a7b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:25.417375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5db68e-d499-4ae5-9e19-d07d2f7a7b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:25.417664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5db68e-d499-4ae5-9e19-d07d2f7a7b86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:25.418424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5db68e-d499-4ae5-9e19-d07d2f7a7b86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:15:25.419307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5db68e-d499-4ae5-9e19-d07d2f7a7b86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:25.421395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5db68e-d499-4ae5-9e19-d07d2f7a7b86 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:25.448391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1095/2177] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:15:25.452880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab87fc-330d-40d9-bd66-012ff9b66311 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:25.467994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab87fc-330d-40d9-bd66-012ff9b66311 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:25.490149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907a509-c3d1-4555-b8ef-6172ee0ea969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:25.492627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907a509-c3d1-4555-b8ef-6172ee0ea969 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:25.494209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907a509-c3d1-4555-b8ef-6172ee0ea969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:25.494738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907a509-c3d1-4555-b8ef-6172ee0ea969 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:25.501778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907a509-c3d1-4555-b8ef-6172ee0ea969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:25.508164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907a509-c3d1-4555-b8ef-6172ee0ea969 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:25.605936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1083/2178] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:15:25 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:25.607237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5db68e-d499-4ae5-9e19-d07d2f7a7b86 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:25.621710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5db68e-d499-4ae5-9e19-d07d2f7a7b86 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:25.803841 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a26bd8-4fb4-4fb6-aaaa-a5e3b0496e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:25.804549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a26bd8-4fb4-4fb6-aaaa-a5e3b0496e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:25.804839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a26bd8-4fb4-4fb6-aaaa-a5e3b0496e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:25.805123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a26bd8-4fb4-4fb6-aaaa-a5e3b0496e6a admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:25.812836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a26bd8-4fb4-4fb6-aaaa-a5e3b0496e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:25.817419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a26bd8-4fb4-4fb6-aaaa-a5e3b0496e6a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:25.857713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1096/2179] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:15:25 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:25.862421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907a509-c3d1-4555-b8ef-6172ee0ea969 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:25.885473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907a509-c3d1-4555-b8ef-6172ee0ea969 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:25.890775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4c5c8-c474-419d-8d8d-bdb49da278c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:25.891540 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4c5c8-c474-419d-8d8d-bdb49da278c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:25.891846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4c5c8-c474-419d-8d8d-bdb49da278c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:25.892130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4c5c8-c474-419d-8d8d-bdb49da278c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:25.892942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4c5c8-c474-419d-8d8d-bdb49da278c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:25.894964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4c5c8-c474-419d-8d8d-bdb49da278c8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:25.911084 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1084/2180] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:25.912124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a26bd8-4fb4-4fb6-aaaa-a5e3b0496e6a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:25.927219 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a26bd8-4fb4-4fb6-aaaa-a5e3b0496e6a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:25.940009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488e58a4-170c-4f08-a224-9c6d92ec292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:25.940859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e58a4-170c-4f08-a224-9c6d92ec292b admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:25.941189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e58a4-170c-4f08-a224-9c6d92ec292b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:25.941600 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488e58a4-170c-4f08-a224-9c6d92ec292b admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:25.942523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e58a4-170c-4f08-a224-9c6d92ec292b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:25.944325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488e58a4-170c-4f08-a224-9c6d92ec292b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:25.952140 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:25.974007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4c5c8-c474-419d-8d8d-bdb49da278c8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:25.985790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1097/2181] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:25.991210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4c5c8-c474-419d-8d8d-bdb49da278c8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:25.999405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f82f20-eabd-46a1-a3c2-a9362a4a6732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:26.000384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f82f20-eabd-46a1-a3c2-a9362a4a6732 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:26.000676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f82f20-eabd-46a1-a3c2-a9362a4a6732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:26.000953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f82f20-eabd-46a1-a3c2-a9362a4a6732 admin admin] PATH_INFO: `/v3/projects/f655aa97e54245238b7d92bdbfc12c2b/users/c256a9802b78488180c583fa96c07e73/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:26.022801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f82f20-eabd-46a1-a3c2-a9362a4a6732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f655aa97e54245238b7d92bdbfc12c2b, user_id=c256a9802b78488180c583fa96c07e73, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:26.025885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f82f20-eabd-46a1-a3c2-a9362a4a6732 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:26.046549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1085/2182] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:15:25 2022] PUT /identity/v3/projects/f655aa97e54245238b7d92bdbfc12c2b/users/c256a9802b78488180c583fa96c07e73/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:26.058259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449cdf8-1207-4210-8d47-a156124f6442 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:26.058709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449cdf8-1207-4210-8d47-a156124f6442 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:26.059097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449cdf8-1207-4210-8d47-a156124f6442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:26.091990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d449cdf8-1207-4210-8d47-a156124f6442 None None] MFA Rules not processed for user `c256a9802b78488180c583fa96c07e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:26.092728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d449cdf8-1207-4210-8d47-a156124f6442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:26.171781 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1098/2183] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:26.183181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f82f20-eabd-46a1-a3c2-a9362a4a6732 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:26.184987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-42f82f20-eabd-46a1-a3c2-a9362a4a6732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:26.253334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f82f20-eabd-46a1-a3c2-a9362a4a6732 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:26.257106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3a95d-04e8-4872-9370-a27c30b2d601 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:26.257752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3a95d-04e8-4872-9370-a27c30b2d601 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:26.258051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3a95d-04e8-4872-9370-a27c30b2d601 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:26.258322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3a95d-04e8-4872-9370-a27c30b2d601 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:26.269869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3a95d-04e8-4872-9370-a27c30b2d601 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:26.271388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3a95d-04e8-4872-9370-a27c30b2d601 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:26.281744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1086/2184] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:15:36.168574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=32 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:36.276892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=3,keystone:SELECT=59,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:38.839211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449cdf8-1207-4210-8d47-a156124f6442 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:38.903810 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449cdf8-1207-4210-8d47-a156124f6442 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:38.908982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:38.910520 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:38.910810 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:38.911117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] PATH_INFO: `/v3/users/c256a9802b78488180c583fa96c07e73` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:38.912847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c256a9802b78488180c583fa96c07e73)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:38.914892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:38.962541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c256a9802b78488180c583fa96c07e73'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:38.963692 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c256a9802b78488180c583fa96c07e73'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:38.964206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c256a9802b78488180c583fa96c07e73'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:38.971528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c256a9802b78488180c583fa96c07e73'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:38.979409 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c256a9802b78488180c583fa96c07e73'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:38.996727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1099/2185] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:38 2022] DELETE /identity/v3/users/c256a9802b78488180c583fa96c07e73 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:39.001228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c457f88-c563-4d82-8623-89f1c1310399 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.001682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c457f88-c563-4d82-8623-89f1c1310399 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.001772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c457f88-c563-4d82-8623-89f1c1310399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.021215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9c457f88-c563-4d82-8623-89f1c1310399 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:39.022414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9c457f88-c563-4d82-8623-89f1c1310399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:39.080641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1087/2186] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:39.090890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:39.092328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:39.148563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13f1a1b-c467-43a7-8ff1-63b8d0922d4c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:39.152170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeade1f-39ae-4866-a4b6-9d7f63bbd58b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:39.152816 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeade1f-39ae-4866-a4b6-9d7f63bbd58b service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.153107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeade1f-39ae-4866-a4b6-9d7f63bbd58b service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.153389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeade1f-39ae-4866-a4b6-9d7f63bbd58b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.163329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeade1f-39ae-4866-a4b6-9d7f63bbd58b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:39.165249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeade1f-39ae-4866-a4b6-9d7f63bbd58b service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:39.174985 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1100/2187] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:39.486143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c457f88-c563-4d82-8623-89f1c1310399 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:39.488421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9c457f88-c563-4d82-8623-89f1c1310399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:39.500275 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c457f88-c563-4d82-8623-89f1c1310399 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:39.504907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:39.505856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.506148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.506443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] PATH_INFO: `/v3/projects/f655aa97e54245238b7d92bdbfc12c2b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.507882 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f655aa97e54245238b7d92bdbfc12c2b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:39.509876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:39.556115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f655aa97e54245238b7d92bdbfc12c2b', 'request_id': 'req-3d2794a0-404c-4d8b-9c40-5516061972fa'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:39.564606 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f655aa97e54245238b7d92bdbfc12c2b', 'request_id': 'req-3d2794a0-404c-4d8b-9c40-5516061972fa'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:39.570821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f655aa97e54245238b7d92bdbfc12c2b', 'request_id': 'req-3d2794a0-404c-4d8b-9c40-5516061972fa'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:39.580943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] The token cache is being invalidate because project f655aa97e54245238b7d92bdbfc12c2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:39.581717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:39.582336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3d2794a0-404c-4d8b-9c40-5516061972fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:39.583871 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1088/2188] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:39 2022] DELETE /identity/v3/projects/f655aa97e54245238b7d92bdbfc12c2b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:39.588853 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf972c-6ede-42c1-a95c-3552cbebd906 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.589141 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf972c-6ede-42c1-a95c-3552cbebd906 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.589419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebf972c-6ede-42c1-a95c-3552cbebd906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.606933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-9ebf972c-6ede-42c1-a95c-3552cbebd906 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:39.608159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9ebf972c-6ede-42c1-a95c-3552cbebd906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:39.625510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:39.669792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1101/2189] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:39.678007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e0c81-d597-49c5-b9f9-57d196c43f39 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.678310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e0c81-d597-49c5-b9f9-57d196c43f39 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.678569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35e0c81-d597-49c5-b9f9-57d196c43f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.697008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b35e0c81-d597-49c5-b9f9-57d196c43f39 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:39.698194 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b35e0c81-d597-49c5-b9f9-57d196c43f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:39.705903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1089/2190] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:39.709894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf972c-6ede-42c1-a95c-3552cbebd906 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:39.719870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebf972c-6ede-42c1-a95c-3552cbebd906 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:39.724501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390f530-70c4-4f68-b4e5-1a5977707c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:39.725163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390f530-70c4-4f68-b4e5-1a5977707c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.725442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390f530-70c4-4f68-b4e5-1a5977707c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.725717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390f530-70c4-4f68-b4e5-1a5977707c61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.726472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390f530-70c4-4f68-b4e5-1a5977707c61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:15:39.726779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390f530-70c4-4f68-b4e5-1a5977707c61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:39.728421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390f530-70c4-4f68-b4e5-1a5977707c61 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:39.740578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1102/2191] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:15:39.744735 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e0c81-d597-49c5-b9f9-57d196c43f39 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:39.754607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35e0c81-d597-49c5-b9f9-57d196c43f39 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:39.759300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfeff7e-8b68-445e-a741-415efeced827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:39.759933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfeff7e-8b68-445e-a741-415efeced827 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.760219 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfeff7e-8b68-445e-a741-415efeced827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.760499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfeff7e-8b68-445e-a741-415efeced827 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.761189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfeff7e-8b68-445e-a741-415efeced827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:39.763126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfeff7e-8b68-445e-a741-415efeced827 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:39.787502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1090/2192] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:15:39 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:39.791340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390f530-70c4-4f68-b4e5-1a5977707c61 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:39.801535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390f530-70c4-4f68-b4e5-1a5977707c61 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:39.855923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f343b9-24f4-4db9-85ca-cafc16d14918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:39.856580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f343b9-24f4-4db9-85ca-cafc16d14918 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.856875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f343b9-24f4-4db9-85ca-cafc16d14918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.857153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f343b9-24f4-4db9-85ca-cafc16d14918 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.857860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f343b9-24f4-4db9-85ca-cafc16d14918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:39.859815 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f343b9-24f4-4db9-85ca-cafc16d14918 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:39.879932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1103/2193] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:15:39 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:15:39.884111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfeff7e-8b68-445e-a741-415efeced827 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:39.894465 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfeff7e-8b68-445e-a741-415efeced827 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:39.899101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8d3c5-04aa-4ea6-a949-bbc80c5b03c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:39.899761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8d3c5-04aa-4ea6-a949-bbc80c5b03c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.900048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8d3c5-04aa-4ea6-a949-bbc80c5b03c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.900329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8d3c5-04aa-4ea6-a949-bbc80c5b03c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.901063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8d3c5-04aa-4ea6-a949-bbc80c5b03c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:39.902696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8d3c5-04aa-4ea6-a949-bbc80c5b03c7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:39.913477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1091/2194] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:15:39.917113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f343b9-24f4-4db9-85ca-cafc16d14918 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:39.929126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f343b9-24f4-4db9-85ca-cafc16d14918 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:39.933833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f1c92-7c83-4ca1-825c-fe130eda9c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:39.934518 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f1c92-7c83-4ca1-825c-fe130eda9c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.934805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f1c92-7c83-4ca1-825c-fe130eda9c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.935116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f1c92-7c83-4ca1-825c-fe130eda9c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.935849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f1c92-7c83-4ca1-825c-fe130eda9c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:39.937478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f1c92-7c83-4ca1-825c-fe130eda9c96 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:39.947685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1104/2195] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:15:39.952193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8d3c5-04aa-4ea6-a949-bbc80c5b03c7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:39.962257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8d3c5-04aa-4ea6-a949-bbc80c5b03c7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:39.967082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2814d78-a81e-40c9-8aee-2a6bbea1e447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:39.967960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2814d78-a81e-40c9-8aee-2a6bbea1e447 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:39.968247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2814d78-a81e-40c9-8aee-2a6bbea1e447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:39.968522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2814d78-a81e-40c9-8aee-2a6bbea1e447 admin admin] PATH_INFO: `/v3/projects/11fa6bcdbc264e98b2cbc60895bd5114/users/6eee7387863a414f92b6ef03150c24ce/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:39.983519 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2814d78-a81e-40c9-8aee-2a6bbea1e447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fa6bcdbc264e98b2cbc60895bd5114, user_id=6eee7387863a414f92b6ef03150c24ce, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:39.986291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2814d78-a81e-40c9-8aee-2a6bbea1e447 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:39.999904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1092/2196] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:15:39 2022] PUT /identity/v3/projects/11fa6bcdbc264e98b2cbc60895bd5114/users/6eee7387863a414f92b6ef03150c24ce/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:40.005544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2859b-2aae-4ef7-9e4e-1e0090a7c027 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:40.005978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2859b-2aae-4ef7-9e4e-1e0090a7c027 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:40.006338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2859b-2aae-4ef7-9e4e-1e0090a7c027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:40.024481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-29b2859b-2aae-4ef7-9e4e-1e0090a7c027 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:40.025688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-29b2859b-2aae-4ef7-9e4e-1e0090a7c027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:40.083505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1105/2197] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:40.093414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2814d78-a81e-40c9-8aee-2a6bbea1e447 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:40.095151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e2814d78-a81e-40c9-8aee-2a6bbea1e447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:40.117587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:40.153103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2814d78-a81e-40c9-8aee-2a6bbea1e447 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:40.156754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6ecc-8fd8-4aab-b0b2-2a3161015dc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:40.157392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6ecc-8fd8-4aab-b0b2-2a3161015dc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:40.157680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6ecc-8fd8-4aab-b0b2-2a3161015dc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:40.157957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef6ecc-8fd8-4aab-b0b2-2a3161015dc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:40.167359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef6ecc-8fd8-4aab-b0b2-2a3161015dc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:40.168814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef6ecc-8fd8-4aab-b0b2-2a3161015dc5 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:40.178574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1093/2198] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:40.770127 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea6a19-f8cb-4441-8918-b98745d76383 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:40.770426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea6a19-f8cb-4441-8918-b98745d76383 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:40.770708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea6a19-f8cb-4441-8918-b98745d76383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:40.797367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-49ea6a19-f8cb-4441-8918-b98745d76383 None None] MFA Rules not processed for user `6eee7387863a414f92b6ef03150c24ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:40.798580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-49ea6a19-f8cb-4441-8918-b98745d76383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:40.815955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:40.875704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1106/2199] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:40.887304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6ecc-8fd8-4aab-b0b2-2a3161015dc5 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:40.888770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-44ef6ecc-8fd8-4aab-b0b2-2a3161015dc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:40.951365 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef6ecc-8fd8-4aab-b0b2-2a3161015dc5 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:40.955056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b67d2c-d106-42d3-adaa-791276c649d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:40.955759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b67d2c-d106-42d3-adaa-791276c649d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:40.956079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b67d2c-d106-42d3-adaa-791276c649d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:40.956925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b67d2c-d106-42d3-adaa-791276c649d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:40.965784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b67d2c-d106-42d3-adaa-791276c649d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:40.967263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b67d2c-d106-42d3-adaa-791276c649d6 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:40.976611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1094/2200] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:15:45.455821 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cde30-dba8-49da-9765-747704322cdd None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:45.456922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cde30-dba8-49da-9765-747704322cdd None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:45.457191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6cde30-dba8-49da-9765-747704322cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:45.484990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-3d6cde30-dba8-49da-9765-747704322cdd None None] MFA Rules not processed for user `6eee7387863a414f92b6ef03150c24ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:45.486193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3d6cde30-dba8-49da-9765-747704322cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:45.511442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1107/2201] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:49.954342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b67d2c-d106-42d3-adaa-791276c649d6 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:49.955810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-25b67d2c-d106-42d3-adaa-791276c649d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:50.025834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b67d2c-d106-42d3-adaa-791276c649d6 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:50.030442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:50.031336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:50.031622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:50.031900 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] PATH_INFO: `/v3/users/b23cf24e2c6748cbacece238b03b6467` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:50.033097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23cf24e2c6748cbacece238b03b6467)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:50.035011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:50.083475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23cf24e2c6748cbacece238b03b6467'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:50.084123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23cf24e2c6748cbacece238b03b6467'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:50.084731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23cf24e2c6748cbacece238b03b6467'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:50.092406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23cf24e2c6748cbacece238b03b6467'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:50.099650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23cf24e2c6748cbacece238b03b6467'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:50.104991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:15:50.121375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1095/2202] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:15:49 2022] DELETE /identity/v3/users/b23cf24e2c6748cbacece238b03b6467 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:15:50.125967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bc4d3-c905-4aa8-b4d7-45284cd8e3e7 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:50.126461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bc4d3-c905-4aa8-b4d7-45284cd8e3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:50.126741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bc4d3-c905-4aa8-b4d7-45284cd8e3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:50.164106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4e6bc4d3-c905-4aa8-b4d7-45284cd8e3e7 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:15:50.165399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4e6bc4d3-c905-4aa8-b4d7-45284cd8e3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:50.235136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1108/2203] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:15:50.244807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:50.246293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:50.346119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5a788-94a1-4e6b-ac28-1dfdc05a9a02 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:50.350095 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f63b4-8e97-441a-b8b6-72ce3a207bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:50.350752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f63b4-8e97-441a-b8b6-72ce3a207bdf service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:50.351062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f63b4-8e97-441a-b8b6-72ce3a207bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:50.351359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f63b4-8e97-441a-b8b6-72ce3a207bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:50.360799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f63b4-8e97-441a-b8b6-72ce3a207bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:50.362765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f63b4-8e97-441a-b8b6-72ce3a207bdf service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:50.372538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1096/2204] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:15:50.658146 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bc4d3-c905-4aa8-b4d7-45284cd8e3e7 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:15:50.659710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4e6bc4d3-c905-4aa8-b4d7-45284cd8e3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:15:50.671919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bc4d3-c905-4aa8-b4d7-45284cd8e3e7 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:15:50.676488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:15:50.677346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:15:50.677629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:15:50.677905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] PATH_INFO: `/v3/projects/664ce4bffd5b44ca827878e6a24ab3f5` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:15:50.679106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=664ce4bffd5b44ca827878e6a24ab3f5)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:15:50.681027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:15:50.727353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664ce4bffd5b44ca827878e6a24ab3f5', 'request_id': 'req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:50.734230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664ce4bffd5b44ca827878e6a24ab3f5', 'request_id': 'req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:50.740707 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664ce4bffd5b44ca827878e6a24ab3f5', 'request_id': 'req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:50.751147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] The token cache is being invalidate because project 664ce4bffd5b44ca827878e6a24ab3f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:15:50.751430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:50.752186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fb0b8f52-af2b-4b9e-8f06-ef3caed2c8fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:15:50.754285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1109/2205] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:15:50 2022] DELETE /identity/v3/projects/664ce4bffd5b44ca827878e6a24ab3f5 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:16:00.368168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=2,keystone:SELECT=47 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:16:00.748060 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:16:09.267128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f63b4-8e97-441a-b8b6-72ce3a207bdf service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:09.270571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c92f63b4-8e97-441a-b8b6-72ce3a207bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:09.284410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f63b4-8e97-441a-b8b6-72ce3a207bdf service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:09.347425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:09.348299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:09.348582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:09.348859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] PATH_INFO: `/v3/users/0b233edb0fd04c7f978aab9eddaf4020` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:09.350071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b233edb0fd04c7f978aab9eddaf4020)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:09.352028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:09.397315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b233edb0fd04c7f978aab9eddaf4020'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:16:09.397896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b233edb0fd04c7f978aab9eddaf4020'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:16:09.398498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b233edb0fd04c7f978aab9eddaf4020'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:16:09.405979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b233edb0fd04c7f978aab9eddaf4020'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:16:09.413323 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b233edb0fd04c7f978aab9eddaf4020'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:16:09.430195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1097/2206] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:16:09 2022] DELETE /identity/v3/users/0b233edb0fd04c7f978aab9eddaf4020 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:16:09.434818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780f645-e531-46cf-b576-32d69141fa91 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:09.435308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780f645-e531-46cf-b576-32d69141fa91 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:09.435456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780f645-e531-46cf-b576-32d69141fa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:09.453379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b780f645-e531-46cf-b576-32d69141fa91 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:16:09.454640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b780f645-e531-46cf-b576-32d69141fa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:09.519379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1110/2207] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:16:09.528913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:09.530327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:09.590496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dba8ff-e412-4d24-9b0c-5b52845734d9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:09.594300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d886d52-5526-4e98-8c0c-d66d57088317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:09.594953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d886d52-5526-4e98-8c0c-d66d57088317 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:09.595268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d886d52-5526-4e98-8c0c-d66d57088317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:09.595547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d886d52-5526-4e98-8c0c-d66d57088317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:09.604942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d886d52-5526-4e98-8c0c-d66d57088317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:09.606356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d886d52-5526-4e98-8c0c-d66d57088317 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:09.616946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1098/2208] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:16:09.930410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780f645-e531-46cf-b576-32d69141fa91 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:09.931975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b780f645-e531-46cf-b576-32d69141fa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:09.944769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780f645-e531-46cf-b576-32d69141fa91 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:09.949618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:09.950577 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:09.950916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:09.951290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] PATH_INFO: `/v3/projects/5a5cb1b4434441a89f7aa54b3fae5625` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:09.952572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a5cb1b4434441a89f7aa54b3fae5625)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:09.954662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:10.001169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a5cb1b4434441a89f7aa54b3fae5625', 'request_id': 'req-8af36e85-101c-46ac-b509-6aedb42c0b27'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:16:10.009296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a5cb1b4434441a89f7aa54b3fae5625', 'request_id': 'req-8af36e85-101c-46ac-b509-6aedb42c0b27'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:16:10.016009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a5cb1b4434441a89f7aa54b3fae5625', 'request_id': 'req-8af36e85-101c-46ac-b509-6aedb42c0b27'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:16:10.027238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] The token cache is being invalidate because project 5a5cb1b4434441a89f7aa54b3fae5625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:16:10.027533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:16:10.028279 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8af36e85-101c-46ac-b509-6aedb42c0b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:16:10.029807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1111/2209] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:16:09 2022] DELETE /identity/v3/projects/5a5cb1b4434441a89f7aa54b3fae5625 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:16:10.034796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bf5d5-2b20-4cb3-96cb-8de14f237e98 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:10.035121 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bf5d5-2b20-4cb3-96cb-8de14f237e98 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:10.035415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bf5d5-2b20-4cb3-96cb-8de14f237e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:10.062211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:16:10.086201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4a9bf5d5-2b20-4cb3-96cb-8de14f237e98 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:16:10.087450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4a9bf5d5-2b20-4cb3-96cb-8de14f237e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:10.145139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1099/2210] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:16:10.153875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bdeaf-1188-4311-a4c3-f0fff515aa21 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:10.154175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bdeaf-1188-4311-a4c3-f0fff515aa21 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:10.154464 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bdeaf-1188-4311-a4c3-f0fff515aa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:10.172501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d60bdeaf-1188-4311-a4c3-f0fff515aa21 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:16:10.173710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d60bdeaf-1188-4311-a4c3-f0fff515aa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:10.181689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1112/2211] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:16:10.185701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bf5d5-2b20-4cb3-96cb-8de14f237e98 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:10.195970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bf5d5-2b20-4cb3-96cb-8de14f237e98 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:10.200598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdec5e-d544-44fc-aeb0-863774ec218d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:10.201258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdec5e-d544-44fc-aeb0-863774ec218d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:10.201547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdec5e-d544-44fc-aeb0-863774ec218d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:10.201828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cdec5e-d544-44fc-aeb0-863774ec218d admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:10.202571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdec5e-d544-44fc-aeb0-863774ec218d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:16:10.202885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdec5e-d544-44fc-aeb0-863774ec218d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:10.204528 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cdec5e-d544-44fc-aeb0-863774ec218d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:10.217160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1100/2212] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:16:10.221519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bdeaf-1188-4311-a4c3-f0fff515aa21 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:10.230758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bdeaf-1188-4311-a4c3-f0fff515aa21 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:10.235510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11ca27-ba8d-4680-84d8-fcc437f91a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:10.236157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11ca27-ba8d-4680-84d8-fcc437f91a02 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:10.236448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11ca27-ba8d-4680-84d8-fcc437f91a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:10.236755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11ca27-ba8d-4680-84d8-fcc437f91a02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:10.237460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11ca27-ba8d-4680-84d8-fcc437f91a02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:10.239405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11ca27-ba8d-4680-84d8-fcc437f91a02 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:10.265797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1113/2213] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:16:10 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:16:10.270436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdec5e-d544-44fc-aeb0-863774ec218d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:10.280596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cdec5e-d544-44fc-aeb0-863774ec218d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:10.338655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3199f7-c19c-4dc6-a087-ab2adb8cd270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:10.339370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3199f7-c19c-4dc6-a087-ab2adb8cd270 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:10.339655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3199f7-c19c-4dc6-a087-ab2adb8cd270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:10.339934 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3199f7-c19c-4dc6-a087-ab2adb8cd270 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:10.340643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3199f7-c19c-4dc6-a087-ab2adb8cd270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:10.342577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3199f7-c19c-4dc6-a087-ab2adb8cd270 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:10.365693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1101/2214] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:16:10 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:16:10.369953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11ca27-ba8d-4680-84d8-fcc437f91a02 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:10.381118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11ca27-ba8d-4680-84d8-fcc437f91a02 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:10.385827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a01d70-542d-4e87-8653-c3f70285dfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:10.386485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a01d70-542d-4e87-8653-c3f70285dfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:10.386769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a01d70-542d-4e87-8653-c3f70285dfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:10.387072 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a01d70-542d-4e87-8653-c3f70285dfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:10.387801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a01d70-542d-4e87-8653-c3f70285dfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:10.389402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a01d70-542d-4e87-8653-c3f70285dfeb admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:10.400514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1114/2215] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:16:10.404362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3199f7-c19c-4dc6-a087-ab2adb8cd270 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:10.414237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3199f7-c19c-4dc6-a087-ab2adb8cd270 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:10.418963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e7413-8876-4ad6-b5e8-450f518dedf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:10.419657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e7413-8876-4ad6-b5e8-450f518dedf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:10.419942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e7413-8876-4ad6-b5e8-450f518dedf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:10.420220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e7413-8876-4ad6-b5e8-450f518dedf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:10.420940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e7413-8876-4ad6-b5e8-450f518dedf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:10.422968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e7413-8876-4ad6-b5e8-450f518dedf5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:10.433407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1102/2216] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:16:10.437545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a01d70-542d-4e87-8653-c3f70285dfeb admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:10.448253 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a01d70-542d-4e87-8653-c3f70285dfeb admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:10.453201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd8371-2294-43c9-a2c0-b080dddd6348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:10.454165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd8371-2294-43c9-a2c0-b080dddd6348 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:10.454523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd8371-2294-43c9-a2c0-b080dddd6348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:10.454868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd8371-2294-43c9-a2c0-b080dddd6348 admin admin] PATH_INFO: `/v3/projects/f4a1a83049f34f45b51cae80f56bc65f/users/cc70414d78304179ad171036ae225bfc/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:10.470474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd8371-2294-43c9-a2c0-b080dddd6348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a1a83049f34f45b51cae80f56bc65f, user_id=cc70414d78304179ad171036ae225bfc, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:10.473177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd8371-2294-43c9-a2c0-b080dddd6348 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:10.487123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1115/2217] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:16:10 2022] PUT /identity/v3/projects/f4a1a83049f34f45b51cae80f56bc65f/users/cc70414d78304179ad171036ae225bfc/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:16:10.492098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e2261-6848-410b-9152-86360a7e0312 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:10.492399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e2261-6848-410b-9152-86360a7e0312 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:10.492688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e2261-6848-410b-9152-86360a7e0312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:10.511271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0e3e2261-6848-410b-9152-86360a7e0312 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:16:10.512482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0e3e2261-6848-410b-9152-86360a7e0312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:10.572108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1103/2218] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:16:10.581533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd8371-2294-43c9-a2c0-b080dddd6348 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:10.583445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c6dd8371-2294-43c9-a2c0-b080dddd6348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:10.605585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:16:10.642596 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd8371-2294-43c9-a2c0-b080dddd6348 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:10.646273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c069b-ca2d-4a25-ad8e-0a1f277c06d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:10.646916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c069b-ca2d-4a25-ad8e-0a1f277c06d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:10.647247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c069b-ca2d-4a25-ad8e-0a1f277c06d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:10.647529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c069b-ca2d-4a25-ad8e-0a1f277c06d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:10.656543 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c069b-ca2d-4a25-ad8e-0a1f277c06d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:10.658017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c069b-ca2d-4a25-ad8e-0a1f277c06d3 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:10.668012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1116/2219] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:16:13.343909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49cf0-1313-40b3-8ac4-063abdc12242 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:13.344427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49cf0-1313-40b3-8ac4-063abdc12242 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:13.344704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b49cf0-1313-40b3-8ac4-063abdc12242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:13.351648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:16:13.365047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c5b49cf0-1313-40b3-8ac4-063abdc12242 None None] MFA Rules not processed for user `6eee7387863a414f92b6ef03150c24ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:16:13.366526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c5b49cf0-1313-40b3-8ac4-063abdc12242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:13.434282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1104/2220] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:16:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:16:15.324395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637de735-e19b-44a4-963f-ba28d6e20cd0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:15.324824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637de735-e19b-44a4-963f-ba28d6e20cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:15.324941 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637de735-e19b-44a4-963f-ba28d6e20cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:15.358766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-637de735-e19b-44a4-963f-ba28d6e20cd0 None None] MFA Rules not processed for user `cc70414d78304179ad171036ae225bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:16:15.360500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-637de735-e19b-44a4-963f-ba28d6e20cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:15.481184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1117/2221] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:16:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:16:15.485672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49cf0-1313-40b3-8ac4-063abdc12242 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:15.505293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b49cf0-1313-40b3-8ac4-063abdc12242 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:15.518965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0d8ba-67c0-4fcb-ab0d-59bc158ff226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:15.519747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0d8ba-67c0-4fcb-ab0d-59bc158ff226 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:15.521277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0d8ba-67c0-4fcb-ab0d-59bc158ff226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:15.521590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0d8ba-67c0-4fcb-ab0d-59bc158ff226 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:15.523770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0d8ba-67c0-4fcb-ab0d-59bc158ff226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:15.532022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0d8ba-67c0-4fcb-ab0d-59bc158ff226 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:15.589553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1105/2222] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:16:15 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:16:15.593666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637de735-e19b-44a4-963f-ba28d6e20cd0 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:15.605805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637de735-e19b-44a4-963f-ba28d6e20cd0 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:15.676165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1fd37-66ca-41f7-a41d-8c5deeeec454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:15.676908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1fd37-66ca-41f7-a41d-8c5deeeec454 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:15.677467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1fd37-66ca-41f7-a41d-8c5deeeec454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:15.677682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb1fd37-66ca-41f7-a41d-8c5deeeec454 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:15.678501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1fd37-66ca-41f7-a41d-8c5deeeec454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:15.683465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb1fd37-66ca-41f7-a41d-8c5deeeec454 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:15.722683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1118/2223] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:16:15 2022] POST /identity/v3/users => generated 295 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:16:15.722886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0d8ba-67c0-4fcb-ab0d-59bc158ff226 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:15.735461 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0d8ba-67c0-4fcb-ab0d-59bc158ff226 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:15.745605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680f616-b5d4-44d9-a95b-253e82796158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:15.746307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680f616-b5d4-44d9-a95b-253e82796158 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:15.746591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680f616-b5d4-44d9-a95b-253e82796158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:15.746867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680f616-b5d4-44d9-a95b-253e82796158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:15.747638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680f616-b5d4-44d9-a95b-253e82796158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:15.750706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680f616-b5d4-44d9-a95b-253e82796158 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:15.769158 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1106/2224] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:16:15.770349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1fd37-66ca-41f7-a41d-8c5deeeec454 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:15.781482 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb1fd37-66ca-41f7-a41d-8c5deeeec454 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:15.789462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8434301-caf2-45e2-9989-8d5176ca3075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:15.790951 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8434301-caf2-45e2-9989-8d5176ca3075 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:15.791616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8434301-caf2-45e2-9989-8d5176ca3075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:15.792115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8434301-caf2-45e2-9989-8d5176ca3075 admin admin] PATH_INFO: `/v3/projects/79248ec3397046b7a6c4b36d9ef464e5/users/c9e6785d9ac2415e990f9d348f4d3b0b/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:15.827581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8434301-caf2-45e2-9989-8d5176ca3075 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79248ec3397046b7a6c4b36d9ef464e5, user_id=c9e6785d9ac2415e990f9d348f4d3b0b, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:15.832708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8434301-caf2-45e2-9989-8d5176ca3075 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:15.858732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1119/2225] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:16:15 2022] PUT /identity/v3/projects/79248ec3397046b7a6c4b36d9ef464e5/users/c9e6785d9ac2415e990f9d348f4d3b0b/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:16:15.862625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680f616-b5d4-44d9-a95b-253e82796158 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:15.885216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680f616-b5d4-44d9-a95b-253e82796158 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:15.962622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d7943-355d-41e2-9278-7ec06016c4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:15.964883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7943-355d-41e2-9278-7ec06016c4ec admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:15.965211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7943-355d-41e2-9278-7ec06016c4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:15.965913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d7943-355d-41e2-9278-7ec06016c4ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:15.966817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7943-355d-41e2-9278-7ec06016c4ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:15.969276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d7943-355d-41e2-9278-7ec06016c4ec admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:15.985585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1107/2226] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:16:15.990664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8434301-caf2-45e2-9989-8d5176ca3075 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:16.003537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8434301-caf2-45e2-9989-8d5176ca3075 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:16.014386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae65c-f0e8-437b-a497-e6051b215e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:16.016538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae65c-f0e8-437b-a497-e6051b215e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:16.016801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae65c-f0e8-437b-a497-e6051b215e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:16.017090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fae65c-f0e8-437b-a497-e6051b215e6e admin admin] PATH_INFO: `/v3/projects/79248ec3397046b7a6c4b36d9ef464e5/users/c9e6785d9ac2415e990f9d348f4d3b0b/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:16.020766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae65c-f0e8-437b-a497-e6051b215e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=79248ec3397046b7a6c4b36d9ef464e5, user_id=c9e6785d9ac2415e990f9d348f4d3b0b, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:16.025412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fae65c-f0e8-437b-a497-e6051b215e6e admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:16.037328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1120/2227] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:16:15 2022] PUT /identity/v3/projects/79248ec3397046b7a6c4b36d9ef464e5/users/c9e6785d9ac2415e990f9d348f4d3b0b/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:16:21.422290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7efd-670f-4eb6-8e69-7f76dc1e242b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:21.422775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7efd-670f-4eb6-8e69-7f76dc1e242b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:21.422881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb7efd-670f-4eb6-8e69-7f76dc1e242b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:21.465427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-bbcb7efd-670f-4eb6-8e69-7f76dc1e242b None None] MFA Rules not processed for user `c9e6785d9ac2415e990f9d348f4d3b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:16:21.466702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bbcb7efd-670f-4eb6-8e69-7f76dc1e242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:21.566868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:16:21.576344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1108/2228] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:16:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:16:21.591371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae65c-f0e8-437b-a497-e6051b215e6e admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:21.593122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-15fae65c-f0e8-437b-a497-e6051b215e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:21.597913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:16:21.778085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fae65c-f0e8-437b-a497-e6051b215e6e admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:21.784344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06debad-6fc4-4805-925c-e51eef2a4127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:21.785009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06debad-6fc4-4805-925c-e51eef2a4127 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:21.785296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06debad-6fc4-4805-925c-e51eef2a4127 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:21.785585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06debad-6fc4-4805-925c-e51eef2a4127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:21.878884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06debad-6fc4-4805-925c-e51eef2a4127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:21.881008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06debad-6fc4-4805-925c-e51eef2a4127 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:21.904174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1121/2229] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:16:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:16:31.249676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7efd-670f-4eb6-8e69-7f76dc1e242b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:31.251600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bbcb7efd-670f-4eb6-8e69-7f76dc1e242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:31.343702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb7efd-670f-4eb6-8e69-7f76dc1e242b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:31.348663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc390c-3971-415e-8a6e-c67404500f5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:31.349315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc390c-3971-415e-8a6e-c67404500f5c service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:31.349598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc390c-3971-415e-8a6e-c67404500f5c service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:31.349874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc390c-3971-415e-8a6e-c67404500f5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:31.351679 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1acc390c-3971-415e-8a6e-c67404500f5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:31.362915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc390c-3971-415e-8a6e-c67404500f5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:31.364525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc390c-3971-415e-8a6e-c67404500f5c service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:31.374653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1109/2230] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:16:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:16:31.887267 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=66 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:16:41.038624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc12338-f185-470b-a715-7f0f51d46d38 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:41.039301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc12338-f185-470b-a715-7f0f51d46d38 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:41.039301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc12338-f185-470b-a715-7f0f51d46d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:41.095993 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-bfc12338-f185-470b-a715-7f0f51d46d38 None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:16:41.097708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bfc12338-f185-470b-a715-7f0f51d46d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:41.162578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1122/2231] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:16:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:16:41.205745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc390c-3971-415e-8a6e-c67404500f5c service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:41.207286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1acc390c-3971-415e-8a6e-c67404500f5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:41.296936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc390c-3971-415e-8a6e-c67404500f5c service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:41.302555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499253c0-8c21-4344-b023-1805e445cfef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:41.303225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499253c0-8c21-4344-b023-1805e445cfef service swift] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:41.303513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499253c0-8c21-4344-b023-1805e445cfef service swift] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:41.303794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499253c0-8c21-4344-b023-1805e445cfef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:41.311898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499253c0-8c21-4344-b023-1805e445cfef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:41.313510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499253c0-8c21-4344-b023-1805e445cfef service swift] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:41.321569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1110/2232] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:16:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:16:50.818906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555119e-0833-4d28-adbf-e22ac5b651d4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:50.819474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555119e-0833-4d28-adbf-e22ac5b651d4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:50.823208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555119e-0833-4d28-adbf-e22ac5b651d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:50.858067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5555119e-0833-4d28-adbf-e22ac5b651d4 None None] MFA Rules not processed for user `cc70414d78304179ad171036ae225bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:16:50.859432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5555119e-0833-4d28-adbf-e22ac5b651d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:50.868004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1123/2233] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:16:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:16:51.318299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=83 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:16:58.512557 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ec91a-5651-4fd0-b7d0-84e4534faa5f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:58.513182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ec91a-5651-4fd0-b7d0-84e4534faa5f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:58.513182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ec91a-5651-4fd0-b7d0-84e4534faa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:58.530570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-eb7ec91a-5651-4fd0-b7d0-84e4534faa5f None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:16:58.531818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eb7ec91a-5651-4fd0-b7d0-84e4534faa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:16:58.539404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1111/2234] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jul 5 15:16:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:16:58.552345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555119e-0833-4d28-adbf-e22ac5b651d4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:16:58.563954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555119e-0833-4d28-adbf-e22ac5b651d4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:16:58.569508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2ce05-3615-471d-9f8e-1341cc55f3b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:16:58.570150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2ce05-3615-471d-9f8e-1341cc55f3b4 service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:16:58.570475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2ce05-3615-471d-9f8e-1341cc55f3b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:16:58.570767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2ce05-3615-471d-9f8e-1341cc55f3b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:16:58.578853 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2ce05-3615-471d-9f8e-1341cc55f3b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:16:58.580527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2ce05-3615-471d-9f8e-1341cc55f3b4 service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:16:58.589144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1124/2235] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:16:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:17:00.908873 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ec91a-5651-4fd0-b7d0-84e4534faa5f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:00.910349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-eb7ec91a-5651-4fd0-b7d0-84e4534faa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:00.990996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ec91a-5651-4fd0-b7d0-84e4534faa5f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:00.995501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:00.996376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:00.996672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:00.996956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] PATH_INFO: `/v3/users/6eee7387863a414f92b6ef03150c24ce` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:00.998173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6eee7387863a414f92b6ef03150c24ce)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:01.000151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:01.046280 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6eee7387863a414f92b6ef03150c24ce'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:17:01.046925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6eee7387863a414f92b6ef03150c24ce'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:17:01.047557 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6eee7387863a414f92b6ef03150c24ce'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:17:01.054662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6eee7387863a414f92b6ef03150c24ce'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:17:01.061755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-68d1e9e7-53c0-44e4-bd62-f1518f58450f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6eee7387863a414f92b6ef03150c24ce'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:17:01.078869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1112/2236] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:17:00 2022] DELETE /identity/v3/users/6eee7387863a414f92b6ef03150c24ce => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:17:01.252541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2ce05-3615-471d-9f8e-1341cc55f3b4 service swift] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:01.264168 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2ce05-3615-471d-9f8e-1341cc55f3b4 service swift] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:01.324186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:01.325443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:01.325846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:01.326133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] PATH_INFO: `/v3/projects/11fa6bcdbc264e98b2cbc60895bd5114` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:01.327369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fa6bcdbc264e98b2cbc60895bd5114)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:01.329274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:01.372635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:01.375798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fa6bcdbc264e98b2cbc60895bd5114', 'request_id': 'req-9fe1aa66-acba-459a-8d40-b582f2139542'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:17:01.383871 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fa6bcdbc264e98b2cbc60895bd5114', 'request_id': 'req-9fe1aa66-acba-459a-8d40-b582f2139542'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:17:01.389829 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fa6bcdbc264e98b2cbc60895bd5114', 'request_id': 'req-9fe1aa66-acba-459a-8d40-b582f2139542'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:17:01.400616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] The token cache is being invalidate because project 11fa6bcdbc264e98b2cbc60895bd5114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:17:01.400904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:17:01.401521 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9fe1aa66-acba-459a-8d40-b582f2139542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:17:01.403420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1125/2237] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:17:01 2022] DELETE /identity/v3/projects/11fa6bcdbc264e98b2cbc60895bd5114 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:17:01.408610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f1215-d403-44f6-9a8b-3c500d6eca6b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:01.408908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f1215-d403-44f6-9a8b-3c500d6eca6b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:01.409192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f1215-d403-44f6-9a8b-3c500d6eca6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:01.427614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e48f1215-d403-44f6-9a8b-3c500d6eca6b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:01.428956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e48f1215-d403-44f6-9a8b-3c500d6eca6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:01.490579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1113/2238] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:17:01.498321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd595a0a-c80b-4816-8f08-4f4f2f193765 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:01.498643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd595a0a-c80b-4816-8f08-4f4f2f193765 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:01.498930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd595a0a-c80b-4816-8f08-4f4f2f193765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:01.518221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-dd595a0a-c80b-4816-8f08-4f4f2f193765 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:01.519691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-dd595a0a-c80b-4816-8f08-4f4f2f193765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:01.528224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1126/2239] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:17:01.532656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f1215-d403-44f6-9a8b-3c500d6eca6b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:01.542831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f1215-d403-44f6-9a8b-3c500d6eca6b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:01.547444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0dd93-d6f5-4943-a68e-b23ff5dbd4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:01.548098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0dd93-d6f5-4943-a68e-b23ff5dbd4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:01.548392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0dd93-d6f5-4943-a68e-b23ff5dbd4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:01.548682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0dd93-d6f5-4943-a68e-b23ff5dbd4c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:01.549984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0dd93-d6f5-4943-a68e-b23ff5dbd4c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:17:01.550296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0dd93-d6f5-4943-a68e-b23ff5dbd4c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:01.552402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0dd93-d6f5-4943-a68e-b23ff5dbd4c8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:01.560643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:01.564409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1114/2240] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:17:01.569355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd595a0a-c80b-4816-8f08-4f4f2f193765 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:01.578616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd595a0a-c80b-4816-8f08-4f4f2f193765 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:01.584501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155cc6d-8951-4967-aa06-fff348cd9df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:01.585215 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155cc6d-8951-4967-aa06-fff348cd9df0 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:01.585513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155cc6d-8951-4967-aa06-fff348cd9df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:01.585797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155cc6d-8951-4967-aa06-fff348cd9df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:01.586498 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155cc6d-8951-4967-aa06-fff348cd9df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:01.588742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155cc6d-8951-4967-aa06-fff348cd9df0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:01.618890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1127/2241] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:17:01 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:17:01.620779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0dd93-d6f5-4943-a68e-b23ff5dbd4c8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:01.638160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0dd93-d6f5-4943-a68e-b23ff5dbd4c8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:01.697217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5a86e-7eb0-441c-95e2-68485ad1b907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:01.697889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5a86e-7eb0-441c-95e2-68485ad1b907 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:01.698177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5a86e-7eb0-441c-95e2-68485ad1b907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:01.698461 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5a86e-7eb0-441c-95e2-68485ad1b907 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:01.699193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5a86e-7eb0-441c-95e2-68485ad1b907 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:01.701120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5a86e-7eb0-441c-95e2-68485ad1b907 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:01.720505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1115/2242] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:17:01 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:17:01.724951 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155cc6d-8951-4967-aa06-fff348cd9df0 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:01.735224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155cc6d-8951-4967-aa06-fff348cd9df0 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:01.740236 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44b8cf-0e7c-4112-994d-c82412582606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:01.740889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44b8cf-0e7c-4112-994d-c82412582606 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:01.741175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44b8cf-0e7c-4112-994d-c82412582606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:01.741452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44b8cf-0e7c-4112-994d-c82412582606 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:01.742178 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44b8cf-0e7c-4112-994d-c82412582606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:01.743822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c44b8cf-0e7c-4112-994d-c82412582606 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:01.754621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1128/2243] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:17:01.758560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5a86e-7eb0-441c-95e2-68485ad1b907 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:01.768993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5a86e-7eb0-441c-95e2-68485ad1b907 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:01.773943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641c1ea-2865-4ed0-98b0-c6c5dfbe4955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:01.774685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c1ea-2865-4ed0-98b0-c6c5dfbe4955 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:01.775061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c1ea-2865-4ed0-98b0-c6c5dfbe4955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:01.775358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641c1ea-2865-4ed0-98b0-c6c5dfbe4955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:01.776125 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641c1ea-2865-4ed0-98b0-c6c5dfbe4955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:01.777871 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641c1ea-2865-4ed0-98b0-c6c5dfbe4955 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:01.788274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1116/2244] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:17:01.792276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44b8cf-0e7c-4112-994d-c82412582606 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:01.802689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44b8cf-0e7c-4112-994d-c82412582606 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:01.807134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef07b4e-4de7-4325-b7b7-38c044d5dbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:01.808004 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef07b4e-4de7-4325-b7b7-38c044d5dbc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:01.808284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef07b4e-4de7-4325-b7b7-38c044d5dbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:01.808557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef07b4e-4de7-4325-b7b7-38c044d5dbc8 admin admin] PATH_INFO: `/v3/projects/2dcd7a16ce904c3bb47be648736adaa6/users/c90fa36eb96a45e9bb3c47cb50757a31/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:01.823216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef07b4e-4de7-4325-b7b7-38c044d5dbc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dcd7a16ce904c3bb47be648736adaa6, user_id=c90fa36eb96a45e9bb3c47cb50757a31, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:01.825862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef07b4e-4de7-4325-b7b7-38c044d5dbc8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:01.838869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1129/2245] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:17:01 2022] PUT /identity/v3/projects/2dcd7a16ce904c3bb47be648736adaa6/users/c90fa36eb96a45e9bb3c47cb50757a31/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:17:01.843854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1e60e-c50c-4db1-9421-23e00d28d0fd None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:01.844242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1e60e-c50c-4db1-9421-23e00d28d0fd None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:01.844585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1e60e-c50c-4db1-9421-23e00d28d0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:01.862276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-feb1e60e-c50c-4db1-9421-23e00d28d0fd None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:01.863509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-feb1e60e-c50c-4db1-9421-23e00d28d0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:01.920901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1117/2246] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:17:01.932154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef07b4e-4de7-4325-b7b7-38c044d5dbc8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:01.933897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4ef07b4e-4de7-4325-b7b7-38c044d5dbc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:01.993182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef07b4e-4de7-4325-b7b7-38c044d5dbc8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:01.999340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c799154-d432-442e-b5fe-688511f17dce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:02.000010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c799154-d432-442e-b5fe-688511f17dce service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:02.000295 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c799154-d432-442e-b5fe-688511f17dce service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:02.000573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c799154-d432-442e-b5fe-688511f17dce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:02.012626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c799154-d432-442e-b5fe-688511f17dce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:02.014543 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c799154-d432-442e-b5fe-688511f17dce service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:02.025109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1130/2247] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:17:06.643517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b74e-9985-4cfd-ac04-0b29eff1b551 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:06.644067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b74e-9985-4cfd-ac04-0b29eff1b551 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:06.644187 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2b74e-9985-4cfd-ac04-0b29eff1b551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:06.674589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0cd2b74e-9985-4cfd-ac04-0b29eff1b551 None None] MFA Rules not processed for user `c90fa36eb96a45e9bb3c47cb50757a31`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:06.675872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0cd2b74e-9985-4cfd-ac04-0b29eff1b551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:06.740836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1118/2248] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:17:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:17:06.745261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c799154-d432-442e-b5fe-688511f17dce service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:06.755364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c799154-d432-442e-b5fe-688511f17dce service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:06.760085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64bf7-a258-4552-9376-4f2be6755e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:06.760743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64bf7-a258-4552-9376-4f2be6755e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:06.761030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64bf7-a258-4552-9376-4f2be6755e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:06.761312 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed64bf7-a258-4552-9376-4f2be6755e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:06.762014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed64bf7-a258-4552-9376-4f2be6755e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:06.763980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed64bf7-a258-4552-9376-4f2be6755e9a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:06.793221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1131/2249] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:17:06 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:17:06.794403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2b74e-9985-4cfd-ac04-0b29eff1b551 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:06.805591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2b74e-9985-4cfd-ac04-0b29eff1b551 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:06.810283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:06.890008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45b86f-3258-4128-ae4c-b31e6de3dbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:06.890832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45b86f-3258-4128-ae4c-b31e6de3dbdc admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:06.891255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45b86f-3258-4128-ae4c-b31e6de3dbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:06.891542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45b86f-3258-4128-ae4c-b31e6de3dbdc admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:06.892469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45b86f-3258-4128-ae4c-b31e6de3dbdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:06.894287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45b86f-3258-4128-ae4c-b31e6de3dbdc admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:06.919805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1119/2250] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:17:06 2022] POST /identity/v3/users => generated 298 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:17:06.921439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64bf7-a258-4552-9376-4f2be6755e9a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:06.937644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed64bf7-a258-4552-9376-4f2be6755e9a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:06.956197 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d1577-a198-4267-9a5b-7b8e95453f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:06.957002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d1577-a198-4267-9a5b-7b8e95453f69 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:06.958136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d1577-a198-4267-9a5b-7b8e95453f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:06.958136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d1577-a198-4267-9a5b-7b8e95453f69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:06.958998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d1577-a198-4267-9a5b-7b8e95453f69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:06.961081 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d1577-a198-4267-9a5b-7b8e95453f69 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:06.989505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1132/2251] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:17:06.991027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45b86f-3258-4128-ae4c-b31e6de3dbdc admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:07.002295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45b86f-3258-4128-ae4c-b31e6de3dbdc admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:07.007141 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a902f94-0b6d-4256-b8f7-bc4d1b07f961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:07.008027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a902f94-0b6d-4256-b8f7-bc4d1b07f961 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:07.008318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a902f94-0b6d-4256-b8f7-bc4d1b07f961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:07.008810 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a902f94-0b6d-4256-b8f7-bc4d1b07f961 admin admin] PATH_INFO: `/v3/projects/18cf5e6fca8a4da498c8724cf0ed8b71/users/ba289dad73ca41dfaf37ac1b673842a6/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:07.027744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a902f94-0b6d-4256-b8f7-bc4d1b07f961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cf5e6fca8a4da498c8724cf0ed8b71, user_id=ba289dad73ca41dfaf37ac1b673842a6, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:07.030750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a902f94-0b6d-4256-b8f7-bc4d1b07f961 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:07.046880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1120/2252] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:17:06 2022] PUT /identity/v3/projects/18cf5e6fca8a4da498c8724cf0ed8b71/users/ba289dad73ca41dfaf37ac1b673842a6/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:17:07.051009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d1577-a198-4267-9a5b-7b8e95453f69 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:07.061647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d1577-a198-4267-9a5b-7b8e95453f69 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:07.104579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:07.129855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c258837-3c85-4e07-937c-4eef2add4eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:07.130550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c258837-3c85-4e07-937c-4eef2add4eba admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:07.130834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c258837-3c85-4e07-937c-4eef2add4eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:07.131221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c258837-3c85-4e07-937c-4eef2add4eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:07.132514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c258837-3c85-4e07-937c-4eef2add4eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:07.133721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c258837-3c85-4e07-937c-4eef2add4eba admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:07.145486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1133/2253] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:17:07.150736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a902f94-0b6d-4256-b8f7-bc4d1b07f961 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:07.188515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a902f94-0b6d-4256-b8f7-bc4d1b07f961 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:07.193678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f66b6-d4ec-4447-abdf-c9131530d283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:07.194582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f66b6-d4ec-4447-abdf-c9131530d283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:07.194863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f66b6-d4ec-4447-abdf-c9131530d283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:07.195198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f66b6-d4ec-4447-abdf-c9131530d283 admin admin] PATH_INFO: `/v3/projects/18cf5e6fca8a4da498c8724cf0ed8b71/users/ba289dad73ca41dfaf37ac1b673842a6/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:07.197157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f66b6-d4ec-4447-abdf-c9131530d283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cf5e6fca8a4da498c8724cf0ed8b71, user_id=ba289dad73ca41dfaf37ac1b673842a6, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:07.199841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f66b6-d4ec-4447-abdf-c9131530d283 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:07.219163 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1121/2254] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:17:07 2022] PUT /identity/v3/projects/18cf5e6fca8a4da498c8724cf0ed8b71/users/ba289dad73ca41dfaf37ac1b673842a6/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:17:12.306058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c258837-3c85-4e07-937c-4eef2add4eba admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:12.309930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5c258837-3c85-4e07-937c-4eef2add4eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:12.413258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c258837-3c85-4e07-937c-4eef2add4eba admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:12.484900 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0755fdf-93d2-43ba-9e8c-bef2df983325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:12.485627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0755fdf-93d2-43ba-9e8c-bef2df983325 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:12.485918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0755fdf-93d2-43ba-9e8c-bef2df983325 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:12.486248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0755fdf-93d2-43ba-9e8c-bef2df983325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:12.488262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f0755fdf-93d2-43ba-9e8c-bef2df983325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:12.570265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0755fdf-93d2-43ba-9e8c-bef2df983325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:12.571754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0755fdf-93d2-43ba-9e8c-bef2df983325 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:12.589788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1134/2255] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:17:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:17:13.514090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1470f-7415-4d04-81d0-dd0fa668dca4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:13.514587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1470f-7415-4d04-81d0-dd0fa668dca4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:13.514980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1470f-7415-4d04-81d0-dd0fa668dca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:13.536143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9df1470f-7415-4d04-81d0-dd0fa668dca4 None None] MFA Rules not processed for user `1514e519960e4404b9a8804a1d9c938f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:13.537690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9df1470f-7415-4d04-81d0-dd0fa668dca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:13.615756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1122/2256] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:17:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:17:14.425659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f7892-5452-4f80-ae0a-7d9dacb668c8 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:14.426151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f7892-5452-4f80-ae0a-7d9dacb668c8 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:14.426387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f7892-5452-4f80-ae0a-7d9dacb668c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:14.533447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-f39f7892-5452-4f80-ae0a-7d9dacb668c8 None None] MFA Rules not processed for user `ba289dad73ca41dfaf37ac1b673842a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:14.534969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f39f7892-5452-4f80-ae0a-7d9dacb668c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:14.581651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:14.643147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1135/2257] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:17:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:17:14.650309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1470f-7415-4d04-81d0-dd0fa668dca4 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:14.651843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9df1470f-7415-4d04-81d0-dd0fa668dca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:14.752241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:14.758246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1470f-7415-4d04-81d0-dd0fa668dca4 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:14.763618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb297a7-86f0-4e6b-ab0e-e55ea20ea6a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450bcd0f22b840c299d08ce9433d6c85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:14.765199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb297a7-86f0-4e6b-ab0e-e55ea20ea6a9 service glance] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:14.765806 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb297a7-86f0-4e6b-ab0e-e55ea20ea6a9 service glance] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:14.766505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb297a7-86f0-4e6b-ab0e-e55ea20ea6a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:14.835791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb297a7-86f0-4e6b-ab0e-e55ea20ea6a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:14.844357 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb297a7-86f0-4e6b-ab0e-e55ea20ea6a9 service glance] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:14.855743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1123/2258] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:17:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:17:14.974499 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff71d7-3ce7-4645-9b52-cec7029b460d None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:14.974869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff71d7-3ce7-4645-9b52-cec7029b460d None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:14.975083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ff71d7-3ce7-4645-9b52-cec7029b460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:14.997611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-15ff71d7-3ce7-4645-9b52-cec7029b460d None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:14.998891 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-15ff71d7-3ce7-4645-9b52-cec7029b460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:15.074994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1136/2259] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:17:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:17:15.087739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb297a7-86f0-4e6b-ab0e-e55ea20ea6a9 service glance] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:15.089797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-deb297a7-86f0-4e6b-ab0e-e55ea20ea6a9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:15.181596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb297a7-86f0-4e6b-ab0e-e55ea20ea6a9 service glance] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:15.199107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496e266-e641-4e16-ae82-d9ec0f1307da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:15.199736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496e266-e641-4e16-ae82-d9ec0f1307da service swift] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:15.200118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496e266-e641-4e16-ae82-d9ec0f1307da service swift] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:15.200497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496e266-e641-4e16-ae82-d9ec0f1307da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:15.210102 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496e266-e641-4e16-ae82-d9ec0f1307da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:15.211806 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496e266-e641-4e16-ae82-d9ec0f1307da service swift] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:15.221141 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1124/2260] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:17:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:17:25.068785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=40 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:25.217467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=59 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:26.777243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c027-1fe6-4aa8-ba7b-0d359aa50f75 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:26.777696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c027-1fe6-4aa8-ba7b-0d359aa50f75 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:26.777780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9c027-1fe6-4aa8-ba7b-0d359aa50f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:26.810817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-acb9c027-1fe6-4aa8-ba7b-0d359aa50f75 None None] MFA Rules not processed for user `c90fa36eb96a45e9bb3c47cb50757a31`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:26.816077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-acb9c027-1fe6-4aa8-ba7b-0d359aa50f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:26.836845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1137/2261] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:17:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:17:27.228562 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3898af5-1fac-4e7f-b821-b20ea4e81988 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:27.229103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3898af5-1fac-4e7f-b821-b20ea4e81988 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:27.229202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3898af5-1fac-4e7f-b821-b20ea4e81988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:27.248237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c3898af5-1fac-4e7f-b821-b20ea4e81988 None None] MFA Rules not processed for user `0187789947064682a8d9e685c2aadb9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:27.250514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c3898af5-1fac-4e7f-b821-b20ea4e81988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:27.318755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1125/2262] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:17:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:17:32.140895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cb5fd-e4cc-4b6a-a538-fab644d9f57c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:32.141827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cb5fd-e4cc-4b6a-a538-fab644d9f57c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:32.142636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7cb5fd-e4cc-4b6a-a538-fab644d9f57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:32.169429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-6e7cb5fd-e4cc-4b6a-a538-fab644d9f57c None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:32.171247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6e7cb5fd-e4cc-4b6a-a538-fab644d9f57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:32.207417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1138/2263] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:17:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:17:32.222357 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3898af5-1fac-4e7f-b821-b20ea4e81988 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:32.234593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3898af5-1fac-4e7f-b821-b20ea4e81988 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:32.240753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db42b53-6b2a-4e3e-849e-bcecd15ee4b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:32.241739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db42b53-6b2a-4e3e-849e-bcecd15ee4b3 service swift] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:32.242086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db42b53-6b2a-4e3e-849e-bcecd15ee4b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:32.242557 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db42b53-6b2a-4e3e-849e-bcecd15ee4b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:32.253044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db42b53-6b2a-4e3e-849e-bcecd15ee4b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:32.254864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db42b53-6b2a-4e3e-849e-bcecd15ee4b3 service swift] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:32.266248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1126/2264] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:17:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:17:42.163472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=6 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:42.262061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=31 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:48.060047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c36562-5cad-4f8f-9c2f-3c639f03a715 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:48.061377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c36562-5cad-4f8f-9c2f-3c639f03a715 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:48.061868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c36562-5cad-4f8f-9c2f-3c639f03a715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:48.083597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b4c36562-5cad-4f8f-9c2f-3c639f03a715 None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:17:48.085014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b4c36562-5cad-4f8f-9c2f-3c639f03a715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:17:48.104767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1139/2265] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:17:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:17:48.133701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db42b53-6b2a-4e3e-849e-bcecd15ee4b3 service swift] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:17:48.147358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db42b53-6b2a-4e3e-849e-bcecd15ee4b3 service swift] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:17:48.156738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3222ca88-a9a7-46a7-9b2c-f7f2acb5b55d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:17:48.158126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222ca88-a9a7-46a7-9b2c-f7f2acb5b55d service swift] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:17:48.160354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222ca88-a9a7-46a7-9b2c-f7f2acb5b55d service swift] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:17:48.161192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3222ca88-a9a7-46a7-9b2c-f7f2acb5b55d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:17:48.172900 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222ca88-a9a7-46a7-9b2c-f7f2acb5b55d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:17:48.174708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3222ca88-a9a7-46a7-9b2c-f7f2acb5b55d service swift] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:17:48.200428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1127/2266] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:17:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:17:58.077449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:17:58.183847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=6 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:18:06.959506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3bb80-29ef-44d1-806c-89293b8a5250 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:06.960109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3bb80-29ef-44d1-806c-89293b8a5250 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:06.960109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3bb80-29ef-44d1-806c-89293b8a5250 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:06.961429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1140/2267] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:18:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:18:06.967641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d8656-d19c-4e6a-9c7b-dc4795e2d64b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:06.968100 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d8656-d19c-4e6a-9c7b-dc4795e2d64b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:06.968349 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d8656-d19c-4e6a-9c7b-dc4795e2d64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:07.004525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ce3d8656-d19c-4e6a-9c7b-dc4795e2d64b None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:18:07.006370 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ce3d8656-d19c-4e6a-9c7b-dc4795e2d64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:18:07.079246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1128/2268] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:18:07.091181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3bb80-29ef-44d1-806c-89293b8a5250 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:18:07.102690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3bb80-29ef-44d1-806c-89293b8a5250 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:18:07.106235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9826e609-996f-4197-8fcd-88e45a0c9862 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450bcd0f22b840c299d08ce9433d6c85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:18:07.106872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826e609-996f-4197-8fcd-88e45a0c9862 service glance] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:07.107193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826e609-996f-4197-8fcd-88e45a0c9862 service glance] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:07.107477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826e609-996f-4197-8fcd-88e45a0c9862 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:07.117533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826e609-996f-4197-8fcd-88e45a0c9862 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:18:07.118901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9826e609-996f-4197-8fcd-88e45a0c9862 service glance] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:18:07.131385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1141/2269] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:18:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:18:07.198060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8cca1-b81c-4a96-a384-75156b3df4c3 None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:07.198972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8cca1-b81c-4a96-a384-75156b3df4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:07.198972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8cca1-b81c-4a96-a384-75156b3df4c3 None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:07.200285 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1129/2270] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:18:07.207705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262ad82-afa0-4a74-afa7-d204aabc2aa1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:07.208571 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262ad82-afa0-4a74-afa7-d204aabc2aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:07.208856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262ad82-afa0-4a74-afa7-d204aabc2aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:07.226854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b262ad82-afa0-4a74-afa7-d204aabc2aa1 None None] MFA Rules not processed for user `cd7fa93466c54632ad95834297410e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:18:07.228100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b262ad82-afa0-4a74-afa7-d204aabc2aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:18:07.234608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1142/2271] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:18:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:18:07.246694 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8cca1-b81c-4a96-a384-75156b3df4c3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:18:07.258141 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8cca1-b81c-4a96-a384-75156b3df4c3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:18:07.261674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359f4d9-ff09-4b8b-94bd-44391348465d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:18:07.262322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f4d9-ff09-4b8b-94bd-44391348465d service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:07.262607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f4d9-ff09-4b8b-94bd-44391348465d service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:07.262878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359f4d9-ff09-4b8b-94bd-44391348465d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:07.269808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359f4d9-ff09-4b8b-94bd-44391348465d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:18:07.271354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359f4d9-ff09-4b8b-94bd-44391348465d service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:18:07.278901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1130/2272] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:18:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:18:07.360957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b792f71-d634-46b5-a5ae-8f134edca1f2 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:07.361280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b792f71-d634-46b5-a5ae-8f134edca1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:07.361570 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b792f71-d634-46b5-a5ae-8f134edca1f2 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:07.362966 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1143/2273] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:18:07.368599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f18bd-147c-4ee0-ae66-1f8c13117b67 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:07.368835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f18bd-147c-4ee0-ae66-1f8c13117b67 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:07.369120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f18bd-147c-4ee0-ae66-1f8c13117b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:07.388605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2f7f18bd-147c-4ee0-ae66-1f8c13117b67 None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:18:07.389802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2f7f18bd-147c-4ee0-ae66-1f8c13117b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:18:07.399000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1131/2274] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:18:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:18:07.409086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b792f71-d634-46b5-a5ae-8f134edca1f2 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:18:07.420189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b792f71-d634-46b5-a5ae-8f134edca1f2 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:18:07.423912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bb98a-ec38-4760-a6c9-54a43daf8ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:18:07.424560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bb98a-ec38-4760-a6c9-54a43daf8ccb service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:07.424849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bb98a-ec38-4760-a6c9-54a43daf8ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:07.425133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bb98a-ec38-4760-a6c9-54a43daf8ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:07.436056 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bb98a-ec38-4760-a6c9-54a43daf8ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:18:07.437496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bb98a-ec38-4760-a6c9-54a43daf8ccb service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:18:07.448828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1144/2275] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:18:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:18:14.727277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f18bd-147c-4ee0-ae66-1f8c13117b67 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:18:14.736481 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f18bd-147c-4ee0-ae66-1f8c13117b67 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:18:14.739990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89075671-2628-48f2-87ef-4e0283510fdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:18:14.740628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89075671-2628-48f2-87ef-4e0283510fdb service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:14.740941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89075671-2628-48f2-87ef-4e0283510fdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:14.741248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89075671-2628-48f2-87ef-4e0283510fdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:14.750045 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89075671-2628-48f2-87ef-4e0283510fdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:18:14.751526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89075671-2628-48f2-87ef-4e0283510fdb service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:18:14.760969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1132/2276] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:18:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:18:15.980116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06054d93-98bf-4e2c-b435-daad2b9d9aed None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:15.980613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06054d93-98bf-4e2c-b435-daad2b9d9aed None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:15.980737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06054d93-98bf-4e2c-b435-daad2b9d9aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:15.998467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-06054d93-98bf-4e2c-b435-daad2b9d9aed None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:18:15.999806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-06054d93-98bf-4e2c-b435-daad2b9d9aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:18:16.007473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1145/2277] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:18:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:18:16.019997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89075671-2628-48f2-87ef-4e0283510fdb service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:18:16.032071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89075671-2628-48f2-87ef-4e0283510fdb service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:18:16.037103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7473b7-25d7-41ad-87d3-79478c04fa50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:18:16.037779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7473b7-25d7-41ad-87d3-79478c04fa50 service swift] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:16.038063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7473b7-25d7-41ad-87d3-79478c04fa50 service swift] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:16.038346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7473b7-25d7-41ad-87d3-79478c04fa50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:16.048134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7473b7-25d7-41ad-87d3-79478c04fa50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:18:16.049889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7473b7-25d7-41ad-87d3-79478c04fa50 service swift] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:18:16.058329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1133/2278] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:18:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:18:16.513243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06054d93-98bf-4e2c-b435-daad2b9d9aed None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:18:16.523646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06054d93-98bf-4e2c-b435-daad2b9d9aed None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:18:16.527301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb4093-771f-4ead-882d-0846f03e60dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:18:16.527957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb4093-771f-4ead-882d-0846f03e60dc service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:16.528243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb4093-771f-4ead-882d-0846f03e60dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:16.528520 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb4093-771f-4ead-882d-0846f03e60dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:16.530309 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0bfb4093-771f-4ead-882d-0846f03e60dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:18:16.544469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb4093-771f-4ead-882d-0846f03e60dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:18:16.545936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfb4093-771f-4ead-882d-0846f03e60dc service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:18:16.557601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1146/2279] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:18:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:18:17.911836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7473b7-25d7-41ad-87d3-79478c04fa50 service swift] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:18:17.922453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7473b7-25d7-41ad-87d3-79478c04fa50 service swift] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:18:17.926142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe70d69-613d-49a2-92ee-cc7847fab0e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:18:17.926794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe70d69-613d-49a2-92ee-cc7847fab0e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:17.927108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe70d69-613d-49a2-92ee-cc7847fab0e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:17.927392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe70d69-613d-49a2-92ee-cc7847fab0e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:17.929712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cfe70d69-613d-49a2-92ee-cc7847fab0e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:18:17.944829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe70d69-613d-49a2-92ee-cc7847fab0e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:18:17.946607 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe70d69-613d-49a2-92ee-cc7847fab0e8 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:18:17.958797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1134/2280] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:18:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:18:26.552133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=24 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:18:27.953997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=64 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:18:31.987843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb4093-771f-4ead-882d-0846f03e60dc service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:18:31.989211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0bfb4093-771f-4ead-882d-0846f03e60dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:18:32.006020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb4093-771f-4ead-882d-0846f03e60dc service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:18:32.012910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebcef2-4315-40a9-be27-6fca15c22095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:18:32.013653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebcef2-4315-40a9-be27-6fca15c22095 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:32.014019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebcef2-4315-40a9-be27-6fca15c22095 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:32.014362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebcef2-4315-40a9-be27-6fca15c22095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:32.016293 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-19ebcef2-4315-40a9-be27-6fca15c22095 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:18:32.075856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebcef2-4315-40a9-be27-6fca15c22095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:18:32.077449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebcef2-4315-40a9-be27-6fca15c22095 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:18:32.087373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1147/2281] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:18:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:18:42.082157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=28 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:18:52.788541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fc11f-d333-408b-94d5-e161d059f65e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:52.789028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fc11f-d333-408b-94d5-e161d059f65e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:52.789149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fc11f-d333-408b-94d5-e161d059f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:52.806173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a29fc11f-d333-408b-94d5-e161d059f65e None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:18:52.807390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a29fc11f-d333-408b-94d5-e161d059f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:18:52.814346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1135/2282] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jul 5 15:18:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:18:52.836466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebcef2-4315-40a9-be27-6fca15c22095 service nova] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:18:52.848434 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebcef2-4315-40a9-be27-6fca15c22095 service nova] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:18:52.853454 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bf440-b4f4-4dd5-ae82-50763279ff2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:18:52.854098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bf440-b4f4-4dd5-ae82-50763279ff2f service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:18:52.854390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bf440-b4f4-4dd5-ae82-50763279ff2f service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:18:52.854673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45bf440-b4f4-4dd5-ae82-50763279ff2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:18:52.862627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bf440-b4f4-4dd5-ae82-50763279ff2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:18:52.864255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45bf440-b4f4-4dd5-ae82-50763279ff2f service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:18:52.872500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1148/2283] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:18:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:19:02.801240 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:19:02.869035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=6 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:19:18.539325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fc11f-d333-408b-94d5-e161d059f65e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:19:18.552520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fc11f-d333-408b-94d5-e161d059f65e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:19:18.557797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac33dc3-283f-4fc8-8026-df0a19a8ce44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:19:18.558441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac33dc3-283f-4fc8-8026-df0a19a8ce44 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:19:18.558731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac33dc3-283f-4fc8-8026-df0a19a8ce44 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:19:18.559012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac33dc3-283f-4fc8-8026-df0a19a8ce44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:19:18.561200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5ac33dc3-283f-4fc8-8026-df0a19a8ce44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:19:18.618921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac33dc3-283f-4fc8-8026-df0a19a8ce44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:19:18.620572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac33dc3-283f-4fc8-8026-df0a19a8ce44 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:19:18.629503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1136/2284] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:19:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:19:24.871855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bf440-b4f4-4dd5-ae82-50763279ff2f service swift] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:19:24.881843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45bf440-b4f4-4dd5-ae82-50763279ff2f service swift] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:19:24.885390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e55ca-094d-4b04-8c94-f1a52c5f1db1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:19:24.886031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e55ca-094d-4b04-8c94-f1a52c5f1db1 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:19:24.886316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e55ca-094d-4b04-8c94-f1a52c5f1db1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:19:24.886593 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e55ca-094d-4b04-8c94-f1a52c5f1db1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:19:24.888866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-342e55ca-094d-4b04-8c94-f1a52c5f1db1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:19:24.898347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e55ca-094d-4b04-8c94-f1a52c5f1db1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:19:24.899800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e55ca-094d-4b04-8c94-f1a52c5f1db1 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:19:24.908214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1149/2285] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:19:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:19:28.625288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=28 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:19:34.904736 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=6 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:19:41.211062 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7ba17-1d81-442f-a443-cf352a150076 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:19:41.211543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7ba17-1d81-442f-a443-cf352a150076 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:19:41.211645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7ba17-1d81-442f-a443-cf352a150076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:19:41.229297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5db7ba17-1d81-442f-a443-cf352a150076 None None] MFA Rules not processed for user `1514e519960e4404b9a8804a1d9c938f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:19:41.230507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5db7ba17-1d81-442f-a443-cf352a150076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:19:41.237875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1137/2286] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:19:47.364427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e55ca-094d-4b04-8c94-f1a52c5f1db1 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:19:47.366039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-342e55ca-094d-4b04-8c94-f1a52c5f1db1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:19:47.383872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e55ca-094d-4b04-8c94-f1a52c5f1db1 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:19:47.390395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf91a6-759a-493f-84c0-e938d3b3f4b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:19:47.391115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf91a6-759a-493f-84c0-e938d3b3f4b1 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:19:47.391408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf91a6-759a-493f-84c0-e938d3b3f4b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:19:47.391690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf91a6-759a-493f-84c0-e938d3b3f4b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:19:47.393547 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5aaf91a6-759a-493f-84c0-e938d3b3f4b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:19:47.406661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf91a6-759a-493f-84c0-e938d3b3f4b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:19:47.408288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf91a6-759a-493f-84c0-e938d3b3f4b1 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:19:47.419115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1150/2287] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:19:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:19:51.070891 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7ba17-1d81-442f-a443-cf352a150076 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:19:51.083786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7ba17-1d81-442f-a443-cf352a150076 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:19:51.089123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a931e-a062-4c6a-8b7d-d462a37095d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:19:51.090023 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a931e-a062-4c6a-8b7d-d462a37095d9 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:19:51.090305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a931e-a062-4c6a-8b7d-d462a37095d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:19:51.091252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a931e-a062-4c6a-8b7d-d462a37095d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:19:51.092994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-554a931e-a062-4c6a-8b7d-d462a37095d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:19:51.102965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a931e-a062-4c6a-8b7d-d462a37095d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:19:51.104580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a931e-a062-4c6a-8b7d-d462a37095d9 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:19:51.113286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1138/2288] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:19:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:19:54.900902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf91a6-759a-493f-84c0-e938d3b3f4b1 service nova] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:19:54.913122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf91a6-759a-493f-84c0-e938d3b3f4b1 service nova] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:19:54.919310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf0cad-f7f5-4ed3-895c-5d31b51fae7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:19:54.919965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf0cad-f7f5-4ed3-895c-5d31b51fae7b service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:19:54.920254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf0cad-f7f5-4ed3-895c-5d31b51fae7b service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:19:54.920535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf0cad-f7f5-4ed3-895c-5d31b51fae7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:19:54.922535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9fbf0cad-f7f5-4ed3-895c-5d31b51fae7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:19:54.935692 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf0cad-f7f5-4ed3-895c-5d31b51fae7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:19:54.946149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf0cad-f7f5-4ed3-895c-5d31b51fae7b service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:19:54.957588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1151/2289] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:19:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:20:00.347753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4cf73-ea58-4f9e-a4e0-d568cd8e6e84 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:20:00.348249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4cf73-ea58-4f9e-a4e0-d568cd8e6e84 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:20:00.348343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4cf73-ea58-4f9e-a4e0-d568cd8e6e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:20:00.365361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2bf4cf73-ea58-4f9e-a4e0-d568cd8e6e84 None None] MFA Rules not processed for user `cc70414d78304179ad171036ae225bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:20:00.366581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2bf4cf73-ea58-4f9e-a4e0-d568cd8e6e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:20:00.419869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1139/2290] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:20:00.966486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf0cad-f7f5-4ed3-895c-5d31b51fae7b service nova] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:20:00.975591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf0cad-f7f5-4ed3-895c-5d31b51fae7b service nova] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:20:00.979408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee84e7-fd8f-4f56-904b-5b555d80bfbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:20:00.980058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee84e7-fd8f-4f56-904b-5b555d80bfbf service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:20:00.980346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee84e7-fd8f-4f56-904b-5b555d80bfbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:20:00.981240 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee84e7-fd8f-4f56-904b-5b555d80bfbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:20:00.983083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-71ee84e7-fd8f-4f56-904b-5b555d80bfbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:20:00.992495 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee84e7-fd8f-4f56-904b-5b555d80bfbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:20:00.993920 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ee84e7-fd8f-4f56-904b-5b555d80bfbf service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:20:01.001725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1152/2291] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:20:03.454564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4cf73-ea58-4f9e-a4e0-d568cd8e6e84 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:20:03.456027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2bf4cf73-ea58-4f9e-a4e0-d568cd8e6e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:20:03.530834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4cf73-ea58-4f9e-a4e0-d568cd8e6e84 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:20:03.535753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d13eca-e962-4127-9a37-272e7106156c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:20:03.536541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d13eca-e962-4127-9a37-272e7106156c service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:20:03.536835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d13eca-e962-4127-9a37-272e7106156c service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:20:03.537111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d13eca-e962-4127-9a37-272e7106156c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:20:03.538874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-49d13eca-e962-4127-9a37-272e7106156c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:20:03.551761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d13eca-e962-4127-9a37-272e7106156c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:20:03.553358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d13eca-e962-4127-9a37-272e7106156c service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:20:03.563277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1140/2292] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:20:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:20:07.953123 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aaf17-bcf6-45b0-b3ab-691d0ffafada None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:20:07.953679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aaf17-bcf6-45b0-b3ab-691d0ffafada None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:20:07.953679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511aaf17-bcf6-45b0-b3ab-691d0ffafada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:20:07.974455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-511aaf17-bcf6-45b0-b3ab-691d0ffafada None None] MFA Rules not processed for user `0187789947064682a8d9e685c2aadb9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:20:07.975910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-511aaf17-bcf6-45b0-b3ab-691d0ffafada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:20:07.984229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1153/2293] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:20:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:20:12.739793 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce32c80-e796-4276-adfc-82b0cb29db21 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:20:12.740307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce32c80-e796-4276-adfc-82b0cb29db21 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:20:12.740566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce32c80-e796-4276-adfc-82b0cb29db21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:20:12.746556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=67 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:20:12.762178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0ce32c80-e796-4276-adfc-82b0cb29db21 None None] MFA Rules not processed for user `8bbc454737084513baa37072cd160e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:20:12.763748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0ce32c80-e796-4276-adfc-82b0cb29db21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:20:12.775864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1141/2294] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:20:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:20:17.968793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=21 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:20:19.122995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511aaf17-bcf6-45b0-b3ab-691d0ffafada None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:20:19.148471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511aaf17-bcf6-45b0-b3ab-691d0ffafada None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:20:19.159512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d95717-f28d-44bc-ae20-f7394134efdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:20:19.160179 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d95717-f28d-44bc-ae20-f7394134efdf service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:20:19.160466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d95717-f28d-44bc-ae20-f7394134efdf service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:20:19.160744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d95717-f28d-44bc-ae20-f7394134efdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:20:19.163721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-00d95717-f28d-44bc-ae20-f7394134efdf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:20:19.182131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d95717-f28d-44bc-ae20-f7394134efdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:20:19.184462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d95717-f28d-44bc-ae20-f7394134efdf service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:20:19.219249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1154/2295] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:20:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:20:22.755570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:20:29.201668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=6 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:20:53.808566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce32c80-e796-4276-adfc-82b0cb29db21 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:20:53.810435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0ce32c80-e796-4276-adfc-82b0cb29db21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:20:53.895844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce32c80-e796-4276-adfc-82b0cb29db21 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:20:53.899562 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b422c-d346-4c9e-aa85-b80ad9a2ed44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:20:53.900217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b422c-d346-4c9e-aa85-b80ad9a2ed44 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:20:53.900504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b422c-d346-4c9e-aa85-b80ad9a2ed44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:20:53.900790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b422c-d346-4c9e-aa85-b80ad9a2ed44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:20:53.903226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-057b422c-d346-4c9e-aa85-b80ad9a2ed44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:20:53.915365 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b422c-d346-4c9e-aa85-b80ad9a2ed44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:20:53.916788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b422c-d346-4c9e-aa85-b80ad9a2ed44 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:20:53.926162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1142/2296] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:20:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:20:55.988804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a10ed-49a7-4b2a-ae9c-e1ea96677733 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:20:55.989317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a10ed-49a7-4b2a-ae9c-e1ea96677733 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:20:55.989426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a10ed-49a7-4b2a-ae9c-e1ea96677733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:20:56.007200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-bb0a10ed-49a7-4b2a-ae9c-e1ea96677733 None None] MFA Rules not processed for user `ef098bbb9ee94bf4aadb4b59516890fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:20:56.008438 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bb0a10ed-49a7-4b2a-ae9c-e1ea96677733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:20:56.016125 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1155/2297] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:20:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:03.921857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=35 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:06.002111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:15.467577 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b422c-d346-4c9e-aa85-b80ad9a2ed44 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:15.477586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b422c-d346-4c9e-aa85-b80ad9a2ed44 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:15.481916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65dc81-61f9-4dd9-8828-f448e92baade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:15.482650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65dc81-61f9-4dd9-8828-f448e92baade service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:15.483029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65dc81-61f9-4dd9-8828-f448e92baade service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:15.483400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65dc81-61f9-4dd9-8828-f448e92baade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:15.485406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cc65dc81-61f9-4dd9-8828-f448e92baade service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:15.560008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65dc81-61f9-4dd9-8828-f448e92baade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:15.561590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65dc81-61f9-4dd9-8828-f448e92baade service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:15.575324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1143/2298] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:18.757805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a10ed-49a7-4b2a-ae9c-e1ea96677733 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:18.763306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bb0a10ed-49a7-4b2a-ae9c-e1ea96677733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:18.781682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a10ed-49a7-4b2a-ae9c-e1ea96677733 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:18.787312 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab4d05-c659-4a64-bb0d-dee54856cd5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:18.788044 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4d05-c659-4a64-bb0d-dee54856cd5b service placement] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:18.788497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4d05-c659-4a64-bb0d-dee54856cd5b service placement] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:18.788962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4d05-c659-4a64-bb0d-dee54856cd5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:18.794421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-17ab4d05-c659-4a64-bb0d-dee54856cd5b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:18.818049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab4d05-c659-4a64-bb0d-dee54856cd5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:18.819825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab4d05-c659-4a64-bb0d-dee54856cd5b service placement] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:18.832666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1156/2299] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:18.835854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65dc81-61f9-4dd9-8828-f448e92baade service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:18.837493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cc65dc81-61f9-4dd9-8828-f448e92baade service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:18.859428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65dc81-61f9-4dd9-8828-f448e92baade service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:18.864790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:18.865924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:18.866365 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:18.866794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] PATH_INFO: `/v3/users/1514e519960e4404b9a8804a1d9c938f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:18.871309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1514e519960e4404b9a8804a1d9c938f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:18.873624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:18.935593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1514e519960e4404b9a8804a1d9c938f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:18.936444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1514e519960e4404b9a8804a1d9c938f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:18.937594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1514e519960e4404b9a8804a1d9c938f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:18.948719 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1514e519960e4404b9a8804a1d9c938f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:18.956398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e8db429c-2a2b-46ab-83be-1e5f9d7b55c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1514e519960e4404b9a8804a1d9c938f'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:18.975908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1144/2300] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:18 2022] DELETE /identity/v3/users/1514e519960e4404b9a8804a1d9c938f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:19.192271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab4d05-c659-4a64-bb0d-dee54856cd5b service placement] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:19.232725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab4d05-c659-4a64-bb0d-dee54856cd5b service placement] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:19.321576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:19.322501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:19.322791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:19.323728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] PATH_INFO: `/v3/projects/5fd444d991ef4ed99512bedf7ef149c1` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:19.324991 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fd444d991ef4ed99512bedf7ef149c1)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:19.326919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:19.375754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fd444d991ef4ed99512bedf7ef149c1', 'request_id': 'req-7b48d28a-8398-4ea5-93d4-338f5bdd7141'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:19.382899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fd444d991ef4ed99512bedf7ef149c1', 'request_id': 'req-7b48d28a-8398-4ea5-93d4-338f5bdd7141'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:19.389683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fd444d991ef4ed99512bedf7ef149c1', 'request_id': 'req-7b48d28a-8398-4ea5-93d4-338f5bdd7141'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:19.399894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] The token cache is being invalidate because project 5fd444d991ef4ed99512bedf7ef149c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:19.400185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:19.400820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7b48d28a-8398-4ea5-93d4-338f5bdd7141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:19.402461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1157/2301] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:19 2022] DELETE /identity/v3/projects/5fd444d991ef4ed99512bedf7ef149c1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:19.407366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b86d4-1826-4f6b-bf4b-bb774ebebed0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:19.407663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b86d4-1826-4f6b-bf4b-bb774ebebed0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:19.407971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b86d4-1826-4f6b-bf4b-bb774ebebed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:19.427446 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-af2b86d4-1826-4f6b-bf4b-bb774ebebed0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:19.428654 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-af2b86d4-1826-4f6b-bf4b-bb774ebebed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:19.489387 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1145/2302] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:19.497751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f2431-f306-476d-a225-ea7246f7b5d5 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:19.498053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f2431-f306-476d-a225-ea7246f7b5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:19.498337 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f2431-f306-476d-a225-ea7246f7b5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:19.517455 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-722f2431-f306-476d-a225-ea7246f7b5d5 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:19.518680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-722f2431-f306-476d-a225-ea7246f7b5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:19.527177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1158/2303] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:19.531441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b86d4-1826-4f6b-bf4b-bb774ebebed0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:19.541598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b86d4-1826-4f6b-bf4b-bb774ebebed0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:19.546885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172917b9-3d6a-42c8-ba57-efea8e031701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:19.547600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172917b9-3d6a-42c8-ba57-efea8e031701 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:19.547890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172917b9-3d6a-42c8-ba57-efea8e031701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:19.548171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172917b9-3d6a-42c8-ba57-efea8e031701 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:19.548921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172917b9-3d6a-42c8-ba57-efea8e031701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:21:19.549236 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172917b9-3d6a-42c8-ba57-efea8e031701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:19.550854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172917b9-3d6a-42c8-ba57-efea8e031701 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:19.552601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:19.569818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1146/2304] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:21:19.571102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f2431-f306-476d-a225-ea7246f7b5d5 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:19.580891 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f2431-f306-476d-a225-ea7246f7b5d5 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:19.585832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a284e-38e9-46d1-8b0f-368ec65cc537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:19.586577 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a284e-38e9-46d1-8b0f-368ec65cc537 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:19.586865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a284e-38e9-46d1-8b0f-368ec65cc537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:19.587233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a284e-38e9-46d1-8b0f-368ec65cc537 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:19.587949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a284e-38e9-46d1-8b0f-368ec65cc537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:19.590189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a284e-38e9-46d1-8b0f-368ec65cc537 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:19.629566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1159/2305] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:19 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:19.632697 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172917b9-3d6a-42c8-ba57-efea8e031701 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:19.643243 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172917b9-3d6a-42c8-ba57-efea8e031701 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:19.701601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828190bd-8fd7-4848-8b8f-f9aff586058b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:19.702276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828190bd-8fd7-4848-8b8f-f9aff586058b admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:19.702563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828190bd-8fd7-4848-8b8f-f9aff586058b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:19.702843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828190bd-8fd7-4848-8b8f-f9aff586058b admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:19.703606 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828190bd-8fd7-4848-8b8f-f9aff586058b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:19.705533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828190bd-8fd7-4848-8b8f-f9aff586058b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:19.727312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1147/2306] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:19 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:19.731280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a284e-38e9-46d1-8b0f-368ec65cc537 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:19.741699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a284e-38e9-46d1-8b0f-368ec65cc537 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:19.746892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36eaf24-bc43-4b7c-bf39-b2caf4edd109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:19.747583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eaf24-bc43-4b7c-bf39-b2caf4edd109 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:19.747903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eaf24-bc43-4b7c-bf39-b2caf4edd109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:19.748149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eaf24-bc43-4b7c-bf39-b2caf4edd109 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:19.748893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36eaf24-bc43-4b7c-bf39-b2caf4edd109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:19.750516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36eaf24-bc43-4b7c-bf39-b2caf4edd109 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:19.761514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1160/2307] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:19.765585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828190bd-8fd7-4848-8b8f-f9aff586058b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:19.775797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828190bd-8fd7-4848-8b8f-f9aff586058b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:19.780647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a997af6-e071-4472-bfcf-87a8337ca72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:19.781312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a997af6-e071-4472-bfcf-87a8337ca72b admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:19.781601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a997af6-e071-4472-bfcf-87a8337ca72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:19.781882 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a997af6-e071-4472-bfcf-87a8337ca72b admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:19.782617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a997af6-e071-4472-bfcf-87a8337ca72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:19.784258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a997af6-e071-4472-bfcf-87a8337ca72b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:19.794380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1148/2308] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:19.798428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36eaf24-bc43-4b7c-bf39-b2caf4edd109 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:19.809358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36eaf24-bc43-4b7c-bf39-b2caf4edd109 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:19.814729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ad7b3-9fb3-4865-9c86-9ab43a1062c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:19.815813 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ad7b3-9fb3-4865-9c86-9ab43a1062c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:19.816183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ad7b3-9fb3-4865-9c86-9ab43a1062c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:19.816637 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ad7b3-9fb3-4865-9c86-9ab43a1062c7 admin admin] PATH_INFO: `/v3/projects/3df78a2f3b1a4ce78a51b70b81ddd837/users/82deee7f5237465ca3979b9de9451056/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:19.833354 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ad7b3-9fb3-4865-9c86-9ab43a1062c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df78a2f3b1a4ce78a51b70b81ddd837, user_id=82deee7f5237465ca3979b9de9451056, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:19.836176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ad7b3-9fb3-4865-9c86-9ab43a1062c7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:19.850969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1161/2309] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:19 2022] PUT /identity/v3/projects/3df78a2f3b1a4ce78a51b70b81ddd837/users/82deee7f5237465ca3979b9de9451056/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:19.861677 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab34cb5-f6fd-4c59-b790-b76adbda0f22 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:19.862143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab34cb5-f6fd-4c59-b790-b76adbda0f22 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:19.862256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab34cb5-f6fd-4c59-b790-b76adbda0f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:19.890837 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-dab34cb5-f6fd-4c59-b790-b76adbda0f22 None None] MFA Rules not processed for user `82deee7f5237465ca3979b9de9451056`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:19.892148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-dab34cb5-f6fd-4c59-b790-b76adbda0f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:19.957894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1149/2310] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:21:19.968940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ad7b3-9fb3-4865-9c86-9ab43a1062c7 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:19.970780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f94ad7b3-9fb3-4865-9c86-9ab43a1062c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:19.977429 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:20.044923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ad7b3-9fb3-4865-9c86-9ab43a1062c7 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:20.050338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b80972-6aaf-4a85-b748-5d8b4f6ed1a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:20.051013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b80972-6aaf-4a85-b748-5d8b4f6ed1a6 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:20.051325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b80972-6aaf-4a85-b748-5d8b4f6ed1a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:20.051601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b80972-6aaf-4a85-b748-5d8b4f6ed1a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:20.059737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b80972-6aaf-4a85-b748-5d8b4f6ed1a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:20.061318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b80972-6aaf-4a85-b748-5d8b4f6ed1a6 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:20.069957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1162/2311] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:20.279220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab34cb5-f6fd-4c59-b790-b76adbda0f22 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:20.348275 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab34cb5-f6fd-4c59-b790-b76adbda0f22 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:20.349912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:20.350782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:20.351081 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:20.351363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] PATH_INFO: `/v3/users/82deee7f5237465ca3979b9de9451056` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:20.352543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82deee7f5237465ca3979b9de9451056)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:20.355313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:20.358133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:20.402716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02de229-aa76-4cb3-9802-54cb3ea168f1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:20.403416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02de229-aa76-4cb3-9802-54cb3ea168f1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:20.403709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02de229-aa76-4cb3-9802-54cb3ea168f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:20.404558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82deee7f5237465ca3979b9de9451056'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:20.405221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82deee7f5237465ca3979b9de9451056'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:20.405836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82deee7f5237465ca3979b9de9451056'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:20.413407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82deee7f5237465ca3979b9de9451056'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:20.421603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8432a55e-7cfb-41d9-a265-2b65ce0d8759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82deee7f5237465ca3979b9de9451056'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:20.422638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d02de229-aa76-4cb3-9802-54cb3ea168f1 None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:20.423889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d02de229-aa76-4cb3-9802-54cb3ea168f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:20.439007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1150/2312] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:20 2022] DELETE /identity/v3/users/82deee7f5237465ca3979b9de9451056 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:20.443405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1b34b-c3f5-414f-aaee-ecf766ec554c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:20.443698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1b34b-c3f5-414f-aaee-ecf766ec554c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:20.443981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1b34b-c3f5-414f-aaee-ecf766ec554c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:20.461250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-69f1b34b-c3f5-414f-aaee-ecf766ec554c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:20.462451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-69f1b34b-c3f5-414f-aaee-ecf766ec554c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:20.482785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1163/2313] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jul 5 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:21:20.525483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1151/2314] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:21:20.532220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02de229-aa76-4cb3-9802-54cb3ea168f1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:20.533750 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d02de229-aa76-4cb3-9802-54cb3ea168f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:20.535020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1b34b-c3f5-414f-aaee-ecf766ec554c None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:20.536366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-69f1b34b-c3f5-414f-aaee-ecf766ec554c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:20.598904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1b34b-c3f5-414f-aaee-ecf766ec554c None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:20.602614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f95167-76cc-4eca-ab54-17196c7e7a6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:20.603310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f95167-76cc-4eca-ab54-17196c7e7a6e service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:20.603437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02de229-aa76-4cb3-9802-54cb3ea168f1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:20.603617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f95167-76cc-4eca-ab54-17196c7e7a6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:20.603872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f95167-76cc-4eca-ab54-17196c7e7a6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:20.613709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f95167-76cc-4eca-ab54-17196c7e7a6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:20.615195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f95167-76cc-4eca-ab54-17196c7e7a6e service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:20.624484 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1152/2315] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:21:20.636768 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:20.664352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7a881-0513-41c9-b5f4-988a07e77443 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:20.664940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7a881-0513-41c9-b5f4-988a07e77443 service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:20.665227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7a881-0513-41c9-b5f4-988a07e77443 service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:20.665509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7a881-0513-41c9-b5f4-988a07e77443 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:20.730252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7a881-0513-41c9-b5f4-988a07e77443 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:20.731921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7a881-0513-41c9-b5f4-988a07e77443 service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:20.741104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1164/2316] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:21:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:21:20.907657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f95167-76cc-4eca-ab54-17196c7e7a6e service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:20.909595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-08f95167-76cc-4eca-ab54-17196c7e7a6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:20.930297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f95167-76cc-4eca-ab54-17196c7e7a6e service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:20.936993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:20.937975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:20.939300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:20.939300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] PATH_INFO: `/v3/projects/3df78a2f3b1a4ce78a51b70b81ddd837` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:20.940522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df78a2f3b1a4ce78a51b70b81ddd837)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:20.942494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:20.974511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:21.008981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df78a2f3b1a4ce78a51b70b81ddd837', 'request_id': 'req-c44cb810-3dc8-47ac-82c5-a10be79c251c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:21.017378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df78a2f3b1a4ce78a51b70b81ddd837', 'request_id': 'req-c44cb810-3dc8-47ac-82c5-a10be79c251c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:21.024653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df78a2f3b1a4ce78a51b70b81ddd837', 'request_id': 'req-c44cb810-3dc8-47ac-82c5-a10be79c251c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:21.037213 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] The token cache is being invalidate because project 3df78a2f3b1a4ce78a51b70b81ddd837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:21.037504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:21.038162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-c44cb810-3dc8-47ac-82c5-a10be79c251c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:21.040488 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1153/2317] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:20 2022] DELETE /identity/v3/projects/3df78a2f3b1a4ce78a51b70b81ddd837 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:21:21.045882 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a061a4e-04fa-4f1b-aee3-814b4fae3e78 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.046310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a061a4e-04fa-4f1b-aee3-814b4fae3e78 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.046466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a061a4e-04fa-4f1b-aee3-814b4fae3e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.065261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5a061a4e-04fa-4f1b-aee3-814b4fae3e78 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:21.066472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5a061a4e-04fa-4f1b-aee3-814b4fae3e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:21.132891 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1165/2318] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:21.140696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933a8ff-bc18-4eba-9348-70b2fceee9fc None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.141079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933a8ff-bc18-4eba-9348-70b2fceee9fc None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.141420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933a8ff-bc18-4eba-9348-70b2fceee9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.160573 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b933a8ff-bc18-4eba-9348-70b2fceee9fc None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:21.161883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b933a8ff-bc18-4eba-9348-70b2fceee9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:21.170693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1154/2319] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:21.174886 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a061a4e-04fa-4f1b-aee3-814b4fae3e78 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.185371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a061a4e-04fa-4f1b-aee3-814b4fae3e78 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.190493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27de4ad-0218-406d-9f17-3c4d6a750010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:21.190670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27de4ad-0218-406d-9f17-3c4d6a750010 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.190942 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27de4ad-0218-406d-9f17-3c4d6a750010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.191246 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27de4ad-0218-406d-9f17-3c4d6a750010 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.192047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27de4ad-0218-406d-9f17-3c4d6a750010 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:21:21.192360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27de4ad-0218-406d-9f17-3c4d6a750010 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:21.194011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27de4ad-0218-406d-9f17-3c4d6a750010 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:21.212016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1166/2320] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:21:21.213331 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933a8ff-bc18-4eba-9348-70b2fceee9fc None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.223390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933a8ff-bc18-4eba-9348-70b2fceee9fc None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.228143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda2d68-8e39-4e49-9237-82801bc463e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:21.228895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda2d68-8e39-4e49-9237-82801bc463e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.229256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda2d68-8e39-4e49-9237-82801bc463e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.230191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda2d68-8e39-4e49-9237-82801bc463e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.230955 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda2d68-8e39-4e49-9237-82801bc463e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:21.233019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda2d68-8e39-4e49-9237-82801bc463e7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:21.259746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1155/2321] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:21 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:21.263915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27de4ad-0218-406d-9f17-3c4d6a750010 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.274155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27de4ad-0218-406d-9f17-3c4d6a750010 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.331067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc317-d768-4390-9232-90f32d861d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:21.331753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc317-d768-4390-9232-90f32d861d6c admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.332039 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc317-d768-4390-9232-90f32d861d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.332319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afc317-d768-4390-9232-90f32d861d6c admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.333069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afc317-d768-4390-9232-90f32d861d6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:21.335365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7afc317-d768-4390-9232-90f32d861d6c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:21.347544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:21.364813 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1167/2322] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:21 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:21.368889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda2d68-8e39-4e49-9237-82801bc463e7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.379763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda2d68-8e39-4e49-9237-82801bc463e7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.384492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184af7d-7fb9-4252-bddb-e98524f23b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:21.385173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184af7d-7fb9-4252-bddb-e98524f23b84 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.385459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184af7d-7fb9-4252-bddb-e98524f23b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.385779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184af7d-7fb9-4252-bddb-e98524f23b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.386517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184af7d-7fb9-4252-bddb-e98524f23b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:21.388167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184af7d-7fb9-4252-bddb-e98524f23b84 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:21.404200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1156/2323] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:21.405505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc317-d768-4390-9232-90f32d861d6c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.417517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afc317-d768-4390-9232-90f32d861d6c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.422653 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdca92a-ce6a-45bf-b665-1fcbe0e275b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:21.423292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdca92a-ce6a-45bf-b665-1fcbe0e275b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.423583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdca92a-ce6a-45bf-b665-1fcbe0e275b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.423854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdca92a-ce6a-45bf-b665-1fcbe0e275b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.424584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdca92a-ce6a-45bf-b665-1fcbe0e275b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:21.426198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdca92a-ce6a-45bf-b665-1fcbe0e275b7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:21.437292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1168/2324] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:21.441436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184af7d-7fb9-4252-bddb-e98524f23b84 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.451765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184af7d-7fb9-4252-bddb-e98524f23b84 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.456772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af2f83-868e-4139-aa34-2e24e333df39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:21.457823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af2f83-868e-4139-aa34-2e24e333df39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.458287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af2f83-868e-4139-aa34-2e24e333df39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.458746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af2f83-868e-4139-aa34-2e24e333df39 admin admin] PATH_INFO: `/v3/projects/5d37c3101cda4985980e1f78b0238d8a/users/7ada903b645f492596cef088fc3211e6/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.474077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af2f83-868e-4139-aa34-2e24e333df39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d37c3101cda4985980e1f78b0238d8a, user_id=7ada903b645f492596cef088fc3211e6, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:21.476759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af2f83-868e-4139-aa34-2e24e333df39 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:21.492549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1157/2325] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:21 2022] PUT /identity/v3/projects/5d37c3101cda4985980e1f78b0238d8a/users/7ada903b645f492596cef088fc3211e6/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:21.504955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a9c5-49d8-4c26-b445-164fb40b1283 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.505474 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a9c5-49d8-4c26-b445-164fb40b1283 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.505706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a9c5-49d8-4c26-b445-164fb40b1283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.535750 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-9bb0a9c5-49d8-4c26-b445-164fb40b1283 None None] MFA Rules not processed for user `7ada903b645f492596cef088fc3211e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:21.537045 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9bb0a9c5-49d8-4c26-b445-164fb40b1283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:21.606590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1169/2326] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:21:21.611109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af2f83-868e-4139-aa34-2e24e333df39 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.683963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af2f83-868e-4139-aa34-2e24e333df39 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.689738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00809c9-56da-40fe-8aa6-d887845b5f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:21.689738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00809c9-56da-40fe-8aa6-d887845b5f5a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.689738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00809c9-56da-40fe-8aa6-d887845b5f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.690075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00809c9-56da-40fe-8aa6-d887845b5f5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.690808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00809c9-56da-40fe-8aa6-d887845b5f5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:21.692790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00809c9-56da-40fe-8aa6-d887845b5f5a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:21.728454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1158/2327] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:21 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:21.729807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0a9c5-49d8-4c26-b445-164fb40b1283 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.741775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0a9c5-49d8-4c26-b445-164fb40b1283 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.804798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462291c2-59c4-440d-9da5-668768ae0874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:21.805490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462291c2-59c4-440d-9da5-668768ae0874 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.805774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462291c2-59c4-440d-9da5-668768ae0874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.806048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462291c2-59c4-440d-9da5-668768ae0874 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.806808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462291c2-59c4-440d-9da5-668768ae0874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:21.808781 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462291c2-59c4-440d-9da5-668768ae0874 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:21.831911 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1170/2328] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:21 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:21:21.836581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00809c9-56da-40fe-8aa6-d887845b5f5a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.850874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00809c9-56da-40fe-8aa6-d887845b5f5a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.857029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bdb6a-4005-408b-a9dc-19528abd81a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:21.857895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bdb6a-4005-408b-a9dc-19528abd81a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.858695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bdb6a-4005-408b-a9dc-19528abd81a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.859115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bdb6a-4005-408b-a9dc-19528abd81a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.859895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bdb6a-4005-408b-a9dc-19528abd81a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:21.861603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bdb6a-4005-408b-a9dc-19528abd81a0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:21.874299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1159/2329] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:21:21.878847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462291c2-59c4-440d-9da5-668768ae0874 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.892047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462291c2-59c4-440d-9da5-668768ae0874 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.896747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444df37-90e7-4645-a716-f3a1d020de58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:21.897638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444df37-90e7-4645-a716-f3a1d020de58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:21.897971 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444df37-90e7-4645-a716-f3a1d020de58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:21.898243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c444df37-90e7-4645-a716-f3a1d020de58 admin admin] PATH_INFO: `/v3/projects/52b0e9c475d943fa82ea7a3389a44686/users/5de71bef92a94dc19aadba1ae451d7a2/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:21.922177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444df37-90e7-4645-a716-f3a1d020de58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b0e9c475d943fa82ea7a3389a44686, user_id=5de71bef92a94dc19aadba1ae451d7a2, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:21.925132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c444df37-90e7-4645-a716-f3a1d020de58 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:21.940574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1171/2330] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:21 2022] PUT /identity/v3/projects/52b0e9c475d943fa82ea7a3389a44686/users/5de71bef92a94dc19aadba1ae451d7a2/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:21.944556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bdb6a-4005-408b-a9dc-19528abd81a0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:21.955599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bdb6a-4005-408b-a9dc-19528abd81a0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:21.975220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:22.033161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac70ade-6aee-4115-af22-5306fc71a61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:22.033910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac70ade-6aee-4115-af22-5306fc71a61c admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:22.034143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac70ade-6aee-4115-af22-5306fc71a61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:22.034468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac70ade-6aee-4115-af22-5306fc71a61c admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:22.035260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac70ade-6aee-4115-af22-5306fc71a61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:22.037025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac70ade-6aee-4115-af22-5306fc71a61c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:22.047829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1160/2331] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:21:22.052318 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444df37-90e7-4645-a716-f3a1d020de58 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:22.065572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c444df37-90e7-4645-a716-f3a1d020de58 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:22.070644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625e77d-430a-45d8-8956-d34220e0dd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:22.071575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625e77d-430a-45d8-8956-d34220e0dd11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:22.071859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625e77d-430a-45d8-8956-d34220e0dd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:22.072133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d625e77d-430a-45d8-8956-d34220e0dd11 admin admin] PATH_INFO: `/v3/projects/52b0e9c475d943fa82ea7a3389a44686/users/5de71bef92a94dc19aadba1ae451d7a2/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:22.074786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d625e77d-430a-45d8-8956-d34220e0dd11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b0e9c475d943fa82ea7a3389a44686, user_id=5de71bef92a94dc19aadba1ae451d7a2, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:22.077496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d625e77d-430a-45d8-8956-d34220e0dd11 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:22.088679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1172/2332] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:22 2022] PUT /identity/v3/projects/52b0e9c475d943fa82ea7a3389a44686/users/5de71bef92a94dc19aadba1ae451d7a2/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:22.099580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e868271-91ae-4238-9654-0d925e73772d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:22.099868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e868271-91ae-4238-9654-0d925e73772d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:22.100154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e868271-91ae-4238-9654-0d925e73772d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:22.130730 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5e868271-91ae-4238-9654-0d925e73772d None None] MFA Rules not processed for user `5de71bef92a94dc19aadba1ae451d7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:22.131990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5e868271-91ae-4238-9654-0d925e73772d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:22.215600 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1161/2333] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:22.226762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625e77d-430a-45d8-8956-d34220e0dd11 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:22.228604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d625e77d-430a-45d8-8956-d34220e0dd11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:22.262211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:22.292697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d625e77d-430a-45d8-8956-d34220e0dd11 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:22.296587 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0cf1e-b115-49b6-abc6-15ad66094c44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:22.297245 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0cf1e-b115-49b6-abc6-15ad66094c44 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:22.297534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0cf1e-b115-49b6-abc6-15ad66094c44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:22.297861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0cf1e-b115-49b6-abc6-15ad66094c44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:22.307518 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0cf1e-b115-49b6-abc6-15ad66094c44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:22.308981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0cf1e-b115-49b6-abc6-15ad66094c44 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:22.319379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1173/2334] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:22.386158 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e868271-91ae-4238-9654-0d925e73772d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:22.451142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e868271-91ae-4238-9654-0d925e73772d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:22.455649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:22.456514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:22.456798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:22.457080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] PATH_INFO: `/v3/users/7ada903b645f492596cef088fc3211e6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:22.458282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ada903b645f492596cef088fc3211e6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:22.460317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:22.506107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:22.519292 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ada903b645f492596cef088fc3211e6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:22.520530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ada903b645f492596cef088fc3211e6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:22.521466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ada903b645f492596cef088fc3211e6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:22.536206 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ada903b645f492596cef088fc3211e6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:22.544207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ada903b645f492596cef088fc3211e6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:22.566775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1162/2335] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:22 2022] DELETE /identity/v3/users/7ada903b645f492596cef088fc3211e6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:22.571305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8ee91-03b3-4d44-9241-d41ad2a7d5b4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:22.571602 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8ee91-03b3-4d44-9241-d41ad2a7d5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:22.571889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8ee91-03b3-4d44-9241-d41ad2a7d5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:22.591627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-69c8ee91-03b3-4d44-9241-d41ad2a7d5b4 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:22.592852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-69c8ee91-03b3-4d44-9241-d41ad2a7d5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:22.660612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1174/2336] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:22.670337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:22.671867 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:22.735927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e30e7c8-32ae-48a6-8ca3-f5407ee7b2d0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:22.740096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc2928-1646-4592-90a2-0cd73f3e9fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:22.740902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc2928-1646-4592-90a2-0cd73f3e9fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:22.741322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc2928-1646-4592-90a2-0cd73f3e9fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:22.741957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afc2928-1646-4592-90a2-0cd73f3e9fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:22.752838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc2928-1646-4592-90a2-0cd73f3e9fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:22.754276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afc2928-1646-4592-90a2-0cd73f3e9fb1 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:22.764625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1163/2337] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:23.087412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8ee91-03b3-4d44-9241-d41ad2a7d5b4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:23.089632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-69c8ee91-03b3-4d44-9241-d41ad2a7d5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:23.102504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8ee91-03b3-4d44-9241-d41ad2a7d5b4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:23.118703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:23.119629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:23.120173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:23.120679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] PATH_INFO: `/v3/projects/5d37c3101cda4985980e1f78b0238d8a` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:23.122026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d37c3101cda4985980e1f78b0238d8a)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:23.124016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:23.184487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d37c3101cda4985980e1f78b0238d8a', 'request_id': 'req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.193235 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d37c3101cda4985980e1f78b0238d8a', 'request_id': 'req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.200655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d37c3101cda4985980e1f78b0238d8a', 'request_id': 'req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.213572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] The token cache is being invalidate because project 5d37c3101cda4985980e1f78b0238d8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:23.214698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.215800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.217389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1175/2338] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:23 2022] DELETE /identity/v3/projects/5d37c3101cda4985980e1f78b0238d8a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:23.221479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc2928-1646-4592-90a2-0cd73f3e9fb1 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:23.232270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afc2928-1646-4592-90a2-0cd73f3e9fb1 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:23.290104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:23.290983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:23.291297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:23.291575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] PATH_INFO: `/v3/users/5de71bef92a94dc19aadba1ae451d7a2` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:23.293000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de71bef92a94dc19aadba1ae451d7a2)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:23.294927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:23.343961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de71bef92a94dc19aadba1ae451d7a2'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.344522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de71bef92a94dc19aadba1ae451d7a2'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.345147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de71bef92a94dc19aadba1ae451d7a2'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.352851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de71bef92a94dc19aadba1ae451d7a2'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.361188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e5cbfa52-c55f-49e2-a854-6c945c4d1d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de71bef92a94dc19aadba1ae451d7a2'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.362466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:23.383325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1164/2339] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:23 2022] DELETE /identity/v3/users/5de71bef92a94dc19aadba1ae451d7a2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:21:23.667717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:23.669186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:23.692022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4b3a3-9e57-4176-ac4a-8fc6f2d5c69b admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:23.750145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:23.761250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:23.762282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:23.762679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:23.763208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] PATH_INFO: `/v3/projects/52b0e9c475d943fa82ea7a3389a44686` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:23.764511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52b0e9c475d943fa82ea7a3389a44686)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:23.766550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:23.827344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b0e9c475d943fa82ea7a3389a44686', 'request_id': 'req-9af47c08-1870-4e40-90e8-fede29bda940'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.844256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b0e9c475d943fa82ea7a3389a44686', 'request_id': 'req-9af47c08-1870-4e40-90e8-fede29bda940'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.852736 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b0e9c475d943fa82ea7a3389a44686', 'request_id': 'req-9af47c08-1870-4e40-90e8-fede29bda940'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.876835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] The token cache is being invalidate because project 52b0e9c475d943fa82ea7a3389a44686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:23.877323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.878416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9af47c08-1870-4e40-90e8-fede29bda940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:23.884665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1176/2340] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:23 2022] DELETE /identity/v3/projects/52b0e9c475d943fa82ea7a3389a44686 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:23.886226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628e97-303a-4d15-a762-0180d52c3547 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:23.886920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628e97-303a-4d15-a762-0180d52c3547 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:23.886920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1628e97-303a-4d15-a762-0180d52c3547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:23.929561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e1628e97-303a-4d15-a762-0180d52c3547 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:23.930876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e1628e97-303a-4d15-a762-0180d52c3547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:24.008011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1165/2341] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:24.017587 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493b5f1-a5f6-48df-b623-3633d546a555 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.017880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493b5f1-a5f6-48df-b623-3633d546a555 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.018163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493b5f1-a5f6-48df-b623-3633d546a555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.044166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-2493b5f1-a5f6-48df-b623-3633d546a555 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:24.045777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2493b5f1-a5f6-48df-b623-3633d546a555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:24.058931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1177/2342] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:24.062997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1628e97-303a-4d15-a762-0180d52c3547 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.072821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1628e97-303a-4d15-a762-0180d52c3547 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.077885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194657e-9863-43b0-8407-9c37bc6743ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.079192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194657e-9863-43b0-8407-9c37bc6743ce admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.079192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194657e-9863-43b0-8407-9c37bc6743ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.079192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194657e-9863-43b0-8407-9c37bc6743ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.079860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194657e-9863-43b0-8407-9c37bc6743ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:21:24.080229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194657e-9863-43b0-8407-9c37bc6743ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.081865 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194657e-9863-43b0-8407-9c37bc6743ce admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.094158 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1166/2343] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:21:24.098451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493b5f1-a5f6-48df-b623-3633d546a555 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.109042 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2493b5f1-a5f6-48df-b623-3633d546a555 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.116154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd395e65-c828-4496-bf4a-2d911e575790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.117417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd395e65-c828-4496-bf4a-2d911e575790 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.118253 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd395e65-c828-4496-bf4a-2d911e575790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.119160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd395e65-c828-4496-bf4a-2d911e575790 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.120142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd395e65-c828-4496-bf4a-2d911e575790 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.122163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd395e65-c828-4496-bf4a-2d911e575790 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.150099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1178/2344] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:24 2022] POST /identity/v3/projects => generated 409 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:24.154210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194657e-9863-43b0-8407-9c37bc6743ce admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.164449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194657e-9863-43b0-8407-9c37bc6743ce admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.237601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113d163-0928-4311-a9b2-23aa948bb2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.238266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113d163-0928-4311-a9b2-23aa948bb2f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.238549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113d163-0928-4311-a9b2-23aa948bb2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.238825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113d163-0928-4311-a9b2-23aa948bb2f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.239560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113d163-0928-4311-a9b2-23aa948bb2f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.241496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113d163-0928-4311-a9b2-23aa948bb2f4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.276683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1167/2345] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:24 2022] POST /identity/v3/users => generated 305 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:24.280402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd395e65-c828-4496-bf4a-2d911e575790 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.291605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd395e65-c828-4496-bf4a-2d911e575790 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.300608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d926d-028e-42f9-ae41-0a36bd62f8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.301301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d926d-028e-42f9-ae41-0a36bd62f8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.301588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d926d-028e-42f9-ae41-0a36bd62f8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.301865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d926d-028e-42f9-ae41-0a36bd62f8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.302605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d926d-028e-42f9-ae41-0a36bd62f8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.304458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d926d-028e-42f9-ae41-0a36bd62f8e7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.316500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1179/2346] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:24.320635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113d163-0928-4311-a9b2-23aa948bb2f4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.331144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113d163-0928-4311-a9b2-23aa948bb2f4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.336129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a391e-2380-45a6-877a-4bb8849ebb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.337025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a391e-2380-45a6-877a-4bb8849ebb63 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.337314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a391e-2380-45a6-877a-4bb8849ebb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.337617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a391e-2380-45a6-877a-4bb8849ebb63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.338348 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a391e-2380-45a6-877a-4bb8849ebb63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.340813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006a391e-2380-45a6-877a-4bb8849ebb63 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.351012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1168/2347] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:24.355803 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d926d-028e-42f9-ae41-0a36bd62f8e7 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.366702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d926d-028e-42f9-ae41-0a36bd62f8e7 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.371419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0a7ba-35a8-4f23-b83a-46560a09184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.372299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0a7ba-35a8-4f23-b83a-46560a09184c admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.372583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0a7ba-35a8-4f23-b83a-46560a09184c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.372875 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe0a7ba-35a8-4f23-b83a-46560a09184c admin admin] PATH_INFO: `/v3/projects/2239d6ce65dc4e6fb5da3dfc780abe64/users/11538a95e29045f2a25964756c3794dc/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.387814 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0a7ba-35a8-4f23-b83a-46560a09184c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2239d6ce65dc4e6fb5da3dfc780abe64, user_id=11538a95e29045f2a25964756c3794dc, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.390483 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe0a7ba-35a8-4f23-b83a-46560a09184c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.405217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1180/2348] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:24 2022] PUT /identity/v3/projects/2239d6ce65dc4e6fb5da3dfc780abe64/users/11538a95e29045f2a25964756c3794dc/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:24.415681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630415b-02bd-4fde-bb00-a9bf070fec33 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.416085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630415b-02bd-4fde-bb00-a9bf070fec33 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.416438 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630415b-02bd-4fde-bb00-a9bf070fec33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.433266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0a7ba-35a8-4f23-b83a-46560a09184c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.435070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8fe0a7ba-35a8-4f23-b83a-46560a09184c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:24.456956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5630415b-02bd-4fde-bb00-a9bf070fec33 None None] MFA Rules not processed for user `11538a95e29045f2a25964756c3794dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:24.457845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5630415b-02bd-4fde-bb00-a9bf070fec33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:24.500128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:24.524183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe0a7ba-35a8-4f23-b83a-46560a09184c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.530534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981afcbc-eea2-4ade-802a-dc20deceb96f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.531224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981afcbc-eea2-4ade-802a-dc20deceb96f service placement] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.531511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981afcbc-eea2-4ade-802a-dc20deceb96f service placement] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.531789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981afcbc-eea2-4ade-802a-dc20deceb96f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.533592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-981afcbc-eea2-4ade-802a-dc20deceb96f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:24.541748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1169/2349] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:21:24.546618 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981afcbc-eea2-4ade-802a-dc20deceb96f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.546761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630415b-02bd-4fde-bb00-a9bf070fec33 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.548261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981afcbc-eea2-4ade-802a-dc20deceb96f service placement] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.558788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1181/2350] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:21:24.617110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630415b-02bd-4fde-bb00-a9bf070fec33 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.621909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526683cc-edc2-4fd9-8b9b-0bbf7ea930e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.622715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526683cc-edc2-4fd9-8b9b-0bbf7ea930e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.623011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526683cc-edc2-4fd9-8b9b-0bbf7ea930e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.623321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526683cc-edc2-4fd9-8b9b-0bbf7ea930e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.624032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526683cc-edc2-4fd9-8b9b-0bbf7ea930e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.626346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526683cc-edc2-4fd9-8b9b-0bbf7ea930e0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.653797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1170/2351] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:24 2022] POST /identity/v3/projects => generated 409 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:24.657848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981afcbc-eea2-4ade-802a-dc20deceb96f service placement] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.669755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981afcbc-eea2-4ade-802a-dc20deceb96f service placement] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.674413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:24.729940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346980ce-f8a1-4192-b3bb-e8d3747f3177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.730615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346980ce-f8a1-4192-b3bb-e8d3747f3177 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.730903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346980ce-f8a1-4192-b3bb-e8d3747f3177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.731215 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346980ce-f8a1-4192-b3bb-e8d3747f3177 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.731924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346980ce-f8a1-4192-b3bb-e8d3747f3177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.733865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346980ce-f8a1-4192-b3bb-e8d3747f3177 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.755525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1182/2352] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:24 2022] POST /identity/v3/users => generated 311 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:24.759625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526683cc-edc2-4fd9-8b9b-0bbf7ea930e0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.769793 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526683cc-edc2-4fd9-8b9b-0bbf7ea930e0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.774375 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb6b4d-5b5a-4039-a8ae-5b197e963f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.775037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb6b4d-5b5a-4039-a8ae-5b197e963f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.775352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb6b4d-5b5a-4039-a8ae-5b197e963f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.775632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeb6b4d-5b5a-4039-a8ae-5b197e963f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.776357 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb6b4d-5b5a-4039-a8ae-5b197e963f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.777948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeb6b4d-5b5a-4039-a8ae-5b197e963f1d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.788612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1171/2353] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:24.793243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346980ce-f8a1-4192-b3bb-e8d3747f3177 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.803258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346980ce-f8a1-4192-b3bb-e8d3747f3177 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.807981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7987d-0314-43cc-a792-baca0917849d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.808924 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7987d-0314-43cc-a792-baca0917849d admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.809136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7987d-0314-43cc-a792-baca0917849d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.809409 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7987d-0314-43cc-a792-baca0917849d admin admin] PATH_INFO: `/v3/projects/8d16326a0f714047b5f48f6678518738/users/301dd38b3390409cb947081a10031d7b/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.826447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7987d-0314-43cc-a792-baca0917849d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d16326a0f714047b5f48f6678518738, user_id=301dd38b3390409cb947081a10031d7b, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.829427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7987d-0314-43cc-a792-baca0917849d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.844877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1183/2354] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:24 2022] PUT /identity/v3/projects/8d16326a0f714047b5f48f6678518738/users/301dd38b3390409cb947081a10031d7b/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:21:24.849026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb6b4d-5b5a-4039-a8ae-5b197e963f1d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.858975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeb6b4d-5b5a-4039-a8ae-5b197e963f1d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.915840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1097906b-917d-4758-b585-7c14b65994ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.916555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097906b-917d-4758-b585-7c14b65994ff admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.916845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097906b-917d-4758-b585-7c14b65994ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.917128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1097906b-917d-4758-b585-7c14b65994ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.917873 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097906b-917d-4758-b585-7c14b65994ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.919541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1097906b-917d-4758-b585-7c14b65994ff admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.929949 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1172/2355] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:24.935071 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7987d-0314-43cc-a792-baca0917849d admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:24.945308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7987d-0314-43cc-a792-baca0917849d admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:24.956269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b526b-c8a7-47ef-803c-fc7e2088cb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:24.957174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b526b-c8a7-47ef-803c-fc7e2088cb74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.957460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b526b-c8a7-47ef-803c-fc7e2088cb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.958026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b526b-c8a7-47ef-803c-fc7e2088cb74 admin admin] PATH_INFO: `/v3/projects/8d16326a0f714047b5f48f6678518738/users/301dd38b3390409cb947081a10031d7b/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:24.960151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b526b-c8a7-47ef-803c-fc7e2088cb74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d16326a0f714047b5f48f6678518738, user_id=301dd38b3390409cb947081a10031d7b, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:24.962876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b526b-c8a7-47ef-803c-fc7e2088cb74 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:24.974636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1184/2356] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:24 2022] PUT /identity/v3/projects/8d16326a0f714047b5f48f6678518738/users/301dd38b3390409cb947081a10031d7b/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:24.985188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa564c8-cfc3-4a8b-8507-a93915e1feda None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:24.985491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa564c8-cfc3-4a8b-8507-a93915e1feda None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:24.985779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa564c8-cfc3-4a8b-8507-a93915e1feda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:25.013172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1aa564c8-cfc3-4a8b-8507-a93915e1feda None None] MFA Rules not processed for user `301dd38b3390409cb947081a10031d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:25.014402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1aa564c8-cfc3-4a8b-8507-a93915e1feda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:25.052424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:25.091989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1173/2357] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:24 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:25.103112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b526b-c8a7-47ef-803c-fc7e2088cb74 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:25.105059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-270b526b-c8a7-47ef-803c-fc7e2088cb74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:25.165667 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b526b-c8a7-47ef-803c-fc7e2088cb74 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:25.169337 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33ccba-c13e-4c1d-b621-688098facc77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:25.169982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ccba-c13e-4c1d-b621-688098facc77 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:25.170328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ccba-c13e-4c1d-b621-688098facc77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:25.170616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33ccba-c13e-4c1d-b621-688098facc77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:25.179903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33ccba-c13e-4c1d-b621-688098facc77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:25.181359 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33ccba-c13e-4c1d-b621-688098facc77 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:25.191714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1185/2358] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:25.345157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa564c8-cfc3-4a8b-8507-a93915e1feda None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:25.356924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa564c8-cfc3-4a8b-8507-a93915e1feda None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:25.360794 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2b52d-7686-46ba-8c54-892efae9dca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:25.361545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2b52d-7686-46ba-8c54-892efae9dca9 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:25.361904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2b52d-7686-46ba-8c54-892efae9dca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:25.362255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2b52d-7686-46ba-8c54-892efae9dca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:25.435347 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2b52d-7686-46ba-8c54-892efae9dca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:25.436912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2b52d-7686-46ba-8c54-892efae9dca9 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:25.447579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1174/2359] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:27.461712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33ccba-c13e-4c1d-b621-688098facc77 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:27.463185 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5c33ccba-c13e-4c1d-b621-688098facc77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:27.527295 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33ccba-c13e-4c1d-b621-688098facc77 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:27.532325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:27.533209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:27.533496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:27.533778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] PATH_INFO: `/v3/users/c90fa36eb96a45e9bb3c47cb50757a31` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:27.534982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c90fa36eb96a45e9bb3c47cb50757a31)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:27.537043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:27.542516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:27.585257 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c90fa36eb96a45e9bb3c47cb50757a31'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:27.585930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c90fa36eb96a45e9bb3c47cb50757a31'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:27.586527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c90fa36eb96a45e9bb3c47cb50757a31'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:27.594196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c90fa36eb96a45e9bb3c47cb50757a31'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:27.601564 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c90fa36eb96a45e9bb3c47cb50757a31'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:27.621110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1186/2360] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:27 2022] DELETE /identity/v3/users/c90fa36eb96a45e9bb3c47cb50757a31 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:27.793027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2b52d-7686-46ba-8c54-892efae9dca9 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:27.804819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2b52d-7686-46ba-8c54-892efae9dca9 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:27.868009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:27.868879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:27.869165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:27.869448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] PATH_INFO: `/v3/projects/2dcd7a16ce904c3bb47be648736adaa6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:27.870668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dcd7a16ce904c3bb47be648736adaa6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:27.872735 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:27.920319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dcd7a16ce904c3bb47be648736adaa6', 'request_id': 'req-06291cbb-3f3a-4944-8891-adfb1ee89abb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:27.927519 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dcd7a16ce904c3bb47be648736adaa6', 'request_id': 'req-06291cbb-3f3a-4944-8891-adfb1ee89abb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:27.934354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dcd7a16ce904c3bb47be648736adaa6', 'request_id': 'req-06291cbb-3f3a-4944-8891-adfb1ee89abb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:27.942499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:27.946471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] The token cache is being invalidate because project 2dcd7a16ce904c3bb47be648736adaa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:27.947315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:27.948456 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:27.951099 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1175/2361] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:27 2022] DELETE /identity/v3/projects/2dcd7a16ce904c3bb47be648736adaa6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:27.955373 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:27.957100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:27.971660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45652d97-b1fc-457b-9317-b4f223d35c00 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:28.030762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:28.031691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.031983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.032266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] PATH_INFO: `/v3/users/ba289dad73ca41dfaf37ac1b673842a6` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.033503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba289dad73ca41dfaf37ac1b673842a6)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:28.035731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:28.085431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba289dad73ca41dfaf37ac1b673842a6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:28.086130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba289dad73ca41dfaf37ac1b673842a6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:28.086763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba289dad73ca41dfaf37ac1b673842a6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:28.094456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba289dad73ca41dfaf37ac1b673842a6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:28.101525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ecd166d5-8dcd-4500-99d4-4868ee0d90bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba289dad73ca41dfaf37ac1b673842a6'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:28.119500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1187/2362] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:27 2022] DELETE /identity/v3/users/ba289dad73ca41dfaf37ac1b673842a6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:28.288415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:28.289870 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:28.302864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06291cbb-3f3a-4944-8891-adfb1ee89abb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:28.364706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:28.365572 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.365853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.366128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] PATH_INFO: `/v3/projects/18cf5e6fca8a4da498c8724cf0ed8b71` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.367325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18cf5e6fca8a4da498c8724cf0ed8b71)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:28.369220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:28.413558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18cf5e6fca8a4da498c8724cf0ed8b71', 'request_id': 'req-81dca945-f8b7-4314-8f5c-f75f6595f4d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:28.420568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18cf5e6fca8a4da498c8724cf0ed8b71', 'request_id': 'req-81dca945-f8b7-4314-8f5c-f75f6595f4d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:28.426635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18cf5e6fca8a4da498c8724cf0ed8b71', 'request_id': 'req-81dca945-f8b7-4314-8f5c-f75f6595f4d6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:28.437775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] The token cache is being invalidate because project 18cf5e6fca8a4da498c8724cf0ed8b71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:28.438080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:28.438706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-81dca945-f8b7-4314-8f5c-f75f6595f4d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:28.440265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1176/2363] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:28 2022] DELETE /identity/v3/projects/18cf5e6fca8a4da498c8724cf0ed8b71 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:28.445154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a7e6-63f0-4841-957a-d4c80602d7d1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.445470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a7e6-63f0-4841-957a-d4c80602d7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.445749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a7e6-63f0-4841-957a-d4c80602d7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.452116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:28.467688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5c08a7e6-63f0-4841-957a-d4c80602d7d1 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:28.468897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5c08a7e6-63f0-4841-957a-d4c80602d7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:28.526465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1188/2364] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:28.534144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e49d5-6f4b-4138-bc24-4ba9aecd7975 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.534434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e49d5-6f4b-4138-bc24-4ba9aecd7975 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.534714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e49d5-6f4b-4138-bc24-4ba9aecd7975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.553285 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ce9e49d5-6f4b-4138-bc24-4ba9aecd7975 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:28.554424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ce9e49d5-6f4b-4138-bc24-4ba9aecd7975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:28.563146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1177/2365] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:28.567241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08a7e6-63f0-4841-957a-d4c80602d7d1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:28.576834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08a7e6-63f0-4841-957a-d4c80602d7d1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:28.581413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1ac22-c23d-49dd-9e50-763d4ec5274a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:28.582057 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1ac22-c23d-49dd-9e50-763d4ec5274a admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.582344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1ac22-c23d-49dd-9e50-763d4ec5274a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.582625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1ac22-c23d-49dd-9e50-763d4ec5274a admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.583391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1ac22-c23d-49dd-9e50-763d4ec5274a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:21:28.583703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1ac22-c23d-49dd-9e50-763d4ec5274a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:28.585301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1ac22-c23d-49dd-9e50-763d4ec5274a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:28.596874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1189/2366] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:21:28.601021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e49d5-6f4b-4138-bc24-4ba9aecd7975 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:28.610764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e49d5-6f4b-4138-bc24-4ba9aecd7975 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:28.615621 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888926c6-e85f-490e-80fd-9df2e328a434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:28.616271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888926c6-e85f-490e-80fd-9df2e328a434 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.616560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888926c6-e85f-490e-80fd-9df2e328a434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.616842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888926c6-e85f-490e-80fd-9df2e328a434 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.617538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888926c6-e85f-490e-80fd-9df2e328a434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:28.619518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888926c6-e85f-490e-80fd-9df2e328a434 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:28.644957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1178/2367] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:28 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:28.649132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1ac22-c23d-49dd-9e50-763d4ec5274a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:28.659006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1ac22-c23d-49dd-9e50-763d4ec5274a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:28.715660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807f93f-27b2-4f5a-9205-79e98fa577d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:28.716363 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807f93f-27b2-4f5a-9205-79e98fa577d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.716652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807f93f-27b2-4f5a-9205-79e98fa577d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.716932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807f93f-27b2-4f5a-9205-79e98fa577d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.717632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807f93f-27b2-4f5a-9205-79e98fa577d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:28.719615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e807f93f-27b2-4f5a-9205-79e98fa577d5 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:28.738509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1190/2368] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:28 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:28.742638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888926c6-e85f-490e-80fd-9df2e328a434 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:28.752961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888926c6-e85f-490e-80fd-9df2e328a434 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:28.757489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c26694-e520-40b4-bda4-5ef1c226959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:28.758139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c26694-e520-40b4-bda4-5ef1c226959c admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.758426 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c26694-e520-40b4-bda4-5ef1c226959c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.758704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c26694-e520-40b4-bda4-5ef1c226959c admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.759507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c26694-e520-40b4-bda4-5ef1c226959c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:28.761101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c26694-e520-40b4-bda4-5ef1c226959c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:28.771258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1179/2369] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:28.774668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807f93f-27b2-4f5a-9205-79e98fa577d5 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:28.784682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e807f93f-27b2-4f5a-9205-79e98fa577d5 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:28.789238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d5140-0b46-4404-9780-28c5041f49ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:28.789879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d5140-0b46-4404-9780-28c5041f49ae admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.790162 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d5140-0b46-4404-9780-28c5041f49ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.790438 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d5140-0b46-4404-9780-28c5041f49ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.791181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d5140-0b46-4404-9780-28c5041f49ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:28.793353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d5140-0b46-4404-9780-28c5041f49ae admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:28.802892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1191/2370] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:28.806827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c26694-e520-40b4-bda4-5ef1c226959c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:28.816751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c26694-e520-40b4-bda4-5ef1c226959c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:28.821407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0ae4d-efb7-4255-9518-56eb90d0d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:28.822277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0ae4d-efb7-4255-9518-56eb90d0d938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.822570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0ae4d-efb7-4255-9518-56eb90d0d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.822852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0ae4d-efb7-4255-9518-56eb90d0d938 admin admin] PATH_INFO: `/v3/projects/32d1debb0ad2409f9b74b0499489f4fc/users/2891dbd54dfb47e4a98d47d2d78b7edf/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.837093 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0ae4d-efb7-4255-9518-56eb90d0d938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d1debb0ad2409f9b74b0499489f4fc, user_id=2891dbd54dfb47e4a98d47d2d78b7edf, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:28.840608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0ae4d-efb7-4255-9518-56eb90d0d938 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:28.853725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1180/2371] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:28 2022] PUT /identity/v3/projects/32d1debb0ad2409f9b74b0499489f4fc/users/2891dbd54dfb47e4a98d47d2d78b7edf/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:28.859333 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1996b5-38f3-46ba-b6f0-d813511287cb None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:28.859634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1996b5-38f3-46ba-b6f0-d813511287cb None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:28.859920 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1996b5-38f3-46ba-b6f0-d813511287cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:28.877188 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0a1996b5-38f3-46ba-b6f0-d813511287cb None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:28.878410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0a1996b5-38f3-46ba-b6f0-d813511287cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:28.940170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1192/2372] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:28.949820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0ae4d-efb7-4255-9518-56eb90d0d938 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:28.951552 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c0f0ae4d-efb7-4255-9518-56eb90d0d938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:28.982760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:29.010644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0ae4d-efb7-4255-9518-56eb90d0d938 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:29.014295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2484b18-c934-4d19-91c3-7028ced8e154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:29.014938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2484b18-c934-4d19-91c3-7028ced8e154 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:29.015264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2484b18-c934-4d19-91c3-7028ced8e154 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:29.015548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2484b18-c934-4d19-91c3-7028ced8e154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:29.024477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2484b18-c934-4d19-91c3-7028ced8e154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:29.025913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2484b18-c934-4d19-91c3-7028ced8e154 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:29.036078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1181/2373] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:32.976489 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab120e-207d-407e-988c-0144d5d9874b None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:32.977015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab120e-207d-407e-988c-0144d5d9874b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:32.977116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab120e-207d-407e-988c-0144d5d9874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:32.997218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:33.008561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-abab120e-207d-407e-988c-0144d5d9874b None None] MFA Rules not processed for user `2891dbd54dfb47e4a98d47d2d78b7edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:33.009713 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-abab120e-207d-407e-988c-0144d5d9874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:33.079583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1193/2374] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:33.084306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2484b18-c934-4d19-91c3-7028ced8e154 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:33.094289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2484b18-c934-4d19-91c3-7028ced8e154 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:33.098906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08604d41-4fc5-410d-94ef-5f0915327705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:33.099679 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08604d41-4fc5-410d-94ef-5f0915327705 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:33.099896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08604d41-4fc5-410d-94ef-5f0915327705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:33.100178 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08604d41-4fc5-410d-94ef-5f0915327705 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:33.100875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08604d41-4fc5-410d-94ef-5f0915327705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:33.102816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08604d41-4fc5-410d-94ef-5f0915327705 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:33.134423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1182/2375] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:33 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:33.148773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab120e-207d-407e-988c-0144d5d9874b None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:33.162661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab120e-207d-407e-988c-0144d5d9874b None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:33.235758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8880cb-6451-4e94-a91e-9d2e088fc84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:33.236461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8880cb-6451-4e94-a91e-9d2e088fc84a admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:33.236749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8880cb-6451-4e94-a91e-9d2e088fc84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:33.237038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8880cb-6451-4e94-a91e-9d2e088fc84a admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:33.237764 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8880cb-6451-4e94-a91e-9d2e088fc84a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:33.241159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8880cb-6451-4e94-a91e-9d2e088fc84a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:33.264630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1194/2376] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:33 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:21:33.269327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08604d41-4fc5-410d-94ef-5f0915327705 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:33.280839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08604d41-4fc5-410d-94ef-5f0915327705 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:33.285840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbad81-a1c5-46a4-9382-641da35f4d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:33.286520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbad81-a1c5-46a4-9382-641da35f4d08 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:33.286812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbad81-a1c5-46a4-9382-641da35f4d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:33.287124 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dbad81-a1c5-46a4-9382-641da35f4d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:33.287869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbad81-a1c5-46a4-9382-641da35f4d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:33.289507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dbad81-a1c5-46a4-9382-641da35f4d08 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:33.301453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1183/2377] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:33.305379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8880cb-6451-4e94-a91e-9d2e088fc84a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:33.315781 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8880cb-6451-4e94-a91e-9d2e088fc84a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:33.320662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fb742-db74-4c3f-9119-6748340fede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:33.321579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fb742-db74-4c3f-9119-6748340fede0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:33.321856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fb742-db74-4c3f-9119-6748340fede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:33.322136 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332fb742-db74-4c3f-9119-6748340fede0 admin admin] PATH_INFO: `/v3/projects/3433b29956f54c6c944f73d80642b663/users/52315700f2a8435abe22ed8a995d6275/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:33.339726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fb742-db74-4c3f-9119-6748340fede0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3433b29956f54c6c944f73d80642b663, user_id=52315700f2a8435abe22ed8a995d6275, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:33.342744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332fb742-db74-4c3f-9119-6748340fede0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:33.357082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1195/2378] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:33 2022] PUT /identity/v3/projects/3433b29956f54c6c944f73d80642b663/users/52315700f2a8435abe22ed8a995d6275/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:33.361066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbad81-a1c5-46a4-9382-641da35f4d08 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:33.371452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dbad81-a1c5-46a4-9382-641da35f4d08 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:33.430166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab1893-09f4-4573-8bd9-f94cfb457608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:33.430851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab1893-09f4-4573-8bd9-f94cfb457608 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:33.431316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab1893-09f4-4573-8bd9-f94cfb457608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:33.431601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab1893-09f4-4573-8bd9-f94cfb457608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:33.432383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab1893-09f4-4573-8bd9-f94cfb457608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:33.433997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aab1893-09f4-4573-8bd9-f94cfb457608 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:33.444635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1184/2379] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:21:33.448862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fb742-db74-4c3f-9119-6748340fede0 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:33.459661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332fb742-db74-4c3f-9119-6748340fede0 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:33.465446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d28cd7-c2f1-4ad3-b6b7-76353e6d0cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:33.466352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d28cd7-c2f1-4ad3-b6b7-76353e6d0cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:33.466636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d28cd7-c2f1-4ad3-b6b7-76353e6d0cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:33.466909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d28cd7-c2f1-4ad3-b6b7-76353e6d0cdd admin admin] PATH_INFO: `/v3/projects/3433b29956f54c6c944f73d80642b663/users/52315700f2a8435abe22ed8a995d6275/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:33.468938 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d28cd7-c2f1-4ad3-b6b7-76353e6d0cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3433b29956f54c6c944f73d80642b663, user_id=52315700f2a8435abe22ed8a995d6275, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:33.471704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d28cd7-c2f1-4ad3-b6b7-76353e6d0cdd admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:33.483468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1196/2380] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:33 2022] PUT /identity/v3/projects/3433b29956f54c6c944f73d80642b663/users/52315700f2a8435abe22ed8a995d6275/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:21:35.098032 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab1893-09f4-4573-8bd9-f94cfb457608 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:35.099856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4aab1893-09f4-4573-8bd9-f94cfb457608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:35.117893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab1893-09f4-4573-8bd9-f94cfb457608 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:35.196451 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:35.200179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:35.200532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:35.200816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] PATH_INFO: `/v3/users/11538a95e29045f2a25964756c3794dc` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:35.205241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=11538a95e29045f2a25964756c3794dc)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:35.208435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:35.269360 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:35.284205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11538a95e29045f2a25964756c3794dc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:35.285157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11538a95e29045f2a25964756c3794dc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:35.285884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11538a95e29045f2a25964756c3794dc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:35.296175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11538a95e29045f2a25964756c3794dc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:35.304264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11538a95e29045f2a25964756c3794dc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:35.324489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1185/2381] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:35 2022] DELETE /identity/v3/users/11538a95e29045f2a25964756c3794dc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:35.329117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1961b1-e576-485c-ab83-8fe5f06c02d0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:35.329941 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1961b1-e576-485c-ab83-8fe5f06c02d0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:35.330408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1961b1-e576-485c-ab83-8fe5f06c02d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:35.352306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-6f1961b1-e576-485c-ab83-8fe5f06c02d0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:35.354366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6f1961b1-e576-485c-ab83-8fe5f06c02d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:35.409486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:35.429504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1197/2382] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:35.444137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:35.446254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:35.509635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd57e4-900f-4da2-8ff8-0752877f02ea admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:35.513740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce808c3-c296-4513-b06b-5fc1dcf7d969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:35.514657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce808c3-c296-4513-b06b-5fc1dcf7d969 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:35.514962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce808c3-c296-4513-b06b-5fc1dcf7d969 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:35.516126 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce808c3-c296-4513-b06b-5fc1dcf7d969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:35.529856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce808c3-c296-4513-b06b-5fc1dcf7d969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:35.530446 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce808c3-c296-4513-b06b-5fc1dcf7d969 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:35.540582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1186/2383] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:21:35.884410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1961b1-e576-485c-ab83-8fe5f06c02d0 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:35.886000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6f1961b1-e576-485c-ab83-8fe5f06c02d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:35.918246 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1961b1-e576-485c-ab83-8fe5f06c02d0 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:35.931435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:35.932884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:35.933173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:35.933456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] PATH_INFO: `/v3/projects/2239d6ce65dc4e6fb5da3dfc780abe64` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:35.934993 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2239d6ce65dc4e6fb5da3dfc780abe64)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:35.938599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:36.064448 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2239d6ce65dc4e6fb5da3dfc780abe64', 'request_id': 'req-dd6d9a05-9852-42da-bc23-07463ca16873'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:36.077023 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2239d6ce65dc4e6fb5da3dfc780abe64', 'request_id': 'req-dd6d9a05-9852-42da-bc23-07463ca16873'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:36.088481 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2239d6ce65dc4e6fb5da3dfc780abe64', 'request_id': 'req-dd6d9a05-9852-42da-bc23-07463ca16873'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:36.120260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] The token cache is being invalidate because project 2239d6ce65dc4e6fb5da3dfc780abe64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:36.120616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:36.122375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:36.130145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1198/2384] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:35 2022] DELETE /identity/v3/projects/2239d6ce65dc4e6fb5da3dfc780abe64 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:36.133069 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce808c3-c296-4513-b06b-5fc1dcf7d969 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:36.144185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce808c3-c296-4513-b06b-5fc1dcf7d969 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:36.223189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:36.225214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:36.225690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:36.226144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] PATH_INFO: `/v3/users/301dd38b3390409cb947081a10031d7b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:36.227640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=301dd38b3390409cb947081a10031d7b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:36.230953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:36.327275 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '301dd38b3390409cb947081a10031d7b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:36.329054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '301dd38b3390409cb947081a10031d7b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:36.330678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '301dd38b3390409cb947081a10031d7b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:36.353624 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '301dd38b3390409cb947081a10031d7b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:36.357066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:36.369412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-469c7f69-eae0-4f20-9586-5e7986d30e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '301dd38b3390409cb947081a10031d7b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:36.418771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1187/2385] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:36 2022] DELETE /identity/v3/users/301dd38b3390409cb947081a10031d7b => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:36.923628 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:36.925679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:36.957148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d9a05-9852-42da-bc23-07463ca16873 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:37.080579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:37.081540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:37.081832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:37.082116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] PATH_INFO: `/v3/projects/8d16326a0f714047b5f48f6678518738` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:37.084884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d16326a0f714047b5f48f6678518738)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:37.086868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:37.153184 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d16326a0f714047b5f48f6678518738', 'request_id': 'req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:37.166952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d16326a0f714047b5f48f6678518738', 'request_id': 'req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:37.176616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d16326a0f714047b5f48f6678518738', 'request_id': 'req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:37.210685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] The token cache is being invalidate because project 8d16326a0f714047b5f48f6678518738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:37.211014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:37.211792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-89f04bf0-b0a6-4dd3-9641-2c7d69f8b70f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:37.218895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1199/2386] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:36 2022] DELETE /identity/v3/projects/8d16326a0f714047b5f48f6678518738 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:37.220339 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34675f-27e9-49fa-a4c9-230b71feeba3 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:37.221190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34675f-27e9-49fa-a4c9-230b71feeba3 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:37.221666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34675f-27e9-49fa-a4c9-230b71feeba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:37.245662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fe34675f-27e9-49fa-a4c9-230b71feeba3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:37.247282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fe34675f-27e9-49fa-a4c9-230b71feeba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:37.345075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739162e3-0080-4fcc-97ac-d578b6c32764 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:37.345463 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739162e3-0080-4fcc-97ac-d578b6c32764 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:37.345854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739162e3-0080-4fcc-97ac-d578b6c32764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:37.348660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1188/2387] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:37.358349 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:37.375172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-739162e3-0080-4fcc-97ac-d578b6c32764 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:37.376771 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-739162e3-0080-4fcc-97ac-d578b6c32764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:37.390804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34675f-27e9-49fa-a4c9-230b71feeba3 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:37.400175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1200/2388] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:37.403468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34675f-27e9-49fa-a4c9-230b71feeba3 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:37.409491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b1c77-9e3b-4ad1-8d7e-7a0bac9c376c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:37.410640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b1c77-9e3b-4ad1-8d7e-7a0bac9c376c admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:37.411191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b1c77-9e3b-4ad1-8d7e-7a0bac9c376c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:37.411708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b1c77-9e3b-4ad1-8d7e-7a0bac9c376c admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:37.412821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b1c77-9e3b-4ad1-8d7e-7a0bac9c376c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:21:37.413498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b1c77-9e3b-4ad1-8d7e-7a0bac9c376c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:37.415859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b1c77-9e3b-4ad1-8d7e-7a0bac9c376c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:37.431125 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1189/2389] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:21:37.436143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739162e3-0080-4fcc-97ac-d578b6c32764 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:37.453168 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739162e3-0080-4fcc-97ac-d578b6c32764 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:37.458644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5bfc-45ef-4621-83be-03719d51b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:37.459366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5bfc-45ef-4621-83be-03719d51b2da admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:37.459700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5bfc-45ef-4621-83be-03719d51b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:37.459927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a5bfc-45ef-4621-83be-03719d51b2da admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:37.460636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a5bfc-45ef-4621-83be-03719d51b2da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:37.463854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0a5bfc-45ef-4621-83be-03719d51b2da admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:37.509568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1201/2390] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:37 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:37.511143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b1c77-9e3b-4ad1-8d7e-7a0bac9c376c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:37.524909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b1c77-9e3b-4ad1-8d7e-7a0bac9c376c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:37.623673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b07d20-d1c1-4fd7-82ed-109657874504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:37.624410 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b07d20-d1c1-4fd7-82ed-109657874504 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:37.624702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b07d20-d1c1-4fd7-82ed-109657874504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:37.624988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b07d20-d1c1-4fd7-82ed-109657874504 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:37.625769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b07d20-d1c1-4fd7-82ed-109657874504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:37.628085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b07d20-d1c1-4fd7-82ed-109657874504 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:37.662176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1190/2391] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:37 2022] POST /identity/v3/users => generated 288 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:37.666015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5bfc-45ef-4621-83be-03719d51b2da admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:37.681921 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a5bfc-45ef-4621-83be-03719d51b2da admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:37.695862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca7c43-0606-4f1b-955d-7140491b0a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:37.696711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca7c43-0606-4f1b-955d-7140491b0a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:37.697065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca7c43-0606-4f1b-955d-7140491b0a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:37.697358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca7c43-0606-4f1b-955d-7140491b0a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:37.698319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca7c43-0606-4f1b-955d-7140491b0a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:37.700576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca7c43-0606-4f1b-955d-7140491b0a3d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:37.727162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b07d20-d1c1-4fd7-82ed-109657874504 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:37.727415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1202/2392] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:37.747587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b07d20-d1c1-4fd7-82ed-109657874504 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:37.758138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2b11b-7fa6-4313-ac48-085afe451da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:37.759284 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2b11b-7fa6-4313-ac48-085afe451da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:37.759732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2b11b-7fa6-4313-ac48-085afe451da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:37.760803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2b11b-7fa6-4313-ac48-085afe451da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:37.761984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2b11b-7fa6-4313-ac48-085afe451da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:37.765209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2b11b-7fa6-4313-ac48-085afe451da2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:37.792657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1191/2393] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:21:37.798002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca7c43-0606-4f1b-955d-7140491b0a3d admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:37.817708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca7c43-0606-4f1b-955d-7140491b0a3d admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:37.828470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b519404-0933-4f2b-ab84-a0acb04893c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:37.830913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b519404-0933-4f2b-ab84-a0acb04893c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:37.830913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b519404-0933-4f2b-ab84-a0acb04893c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:37.831181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b519404-0933-4f2b-ab84-a0acb04893c8 admin admin] PATH_INFO: `/v3/projects/7fd71b0ea6a24683b4343a6a1a288a8e/users/758655f1a1464321869d8044a9dcc263/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:37.855142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b519404-0933-4f2b-ab84-a0acb04893c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fd71b0ea6a24683b4343a6a1a288a8e, user_id=758655f1a1464321869d8044a9dcc263, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:37.858261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b519404-0933-4f2b-ab84-a0acb04893c8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:37.887876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1203/2394] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:37 2022] PUT /identity/v3/projects/7fd71b0ea6a24683b4343a6a1a288a8e/users/758655f1a1464321869d8044a9dcc263/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:37.899932 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8771e-39f4-42c8-8e80-4a7ebc894212 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:37.900508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8771e-39f4-42c8-8e80-4a7ebc894212 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:37.901493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8771e-39f4-42c8-8e80-4a7ebc894212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:37.962968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-aef8771e-39f4-42c8-8e80-4a7ebc894212 None None] MFA Rules not processed for user `758655f1a1464321869d8044a9dcc263`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:37.964828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-aef8771e-39f4-42c8-8e80-4a7ebc894212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:38.024961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:38.090270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1192/2395] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:38.092075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b519404-0933-4f2b-ab84-a0acb04893c8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:38.176337 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b519404-0933-4f2b-ab84-a0acb04893c8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:38.183238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95302a-237e-4d11-999e-9b924ed961ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:38.184062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95302a-237e-4d11-999e-9b924ed961ab admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.184548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95302a-237e-4d11-999e-9b924ed961ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.185047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95302a-237e-4d11-999e-9b924ed961ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.187006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95302a-237e-4d11-999e-9b924ed961ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:38.189582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95302a-237e-4d11-999e-9b924ed961ab admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:38.234343 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1204/2396] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:38 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:38.238097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8771e-39f4-42c8-8e80-4a7ebc894212 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:38.250807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8771e-39f4-42c8-8e80-4a7ebc894212 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:38.333374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa90c13-d51d-44e2-af60-dfad922f312f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:38.333374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa90c13-d51d-44e2-af60-dfad922f312f admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.333374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa90c13-d51d-44e2-af60-dfad922f312f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.333725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa90c13-d51d-44e2-af60-dfad922f312f admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.334312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa90c13-d51d-44e2-af60-dfad922f312f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:38.336311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa90c13-d51d-44e2-af60-dfad922f312f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:38.360461 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1193/2397] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:38 2022] POST /identity/v3/users => generated 289 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:38.364259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95302a-237e-4d11-999e-9b924ed961ab admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:38.380168 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95302a-237e-4d11-999e-9b924ed961ab admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:38.389056 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89620a-34f6-4148-82b5-c71219e3af21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:38.389867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89620a-34f6-4148-82b5-c71219e3af21 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.390225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89620a-34f6-4148-82b5-c71219e3af21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.390578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f89620a-34f6-4148-82b5-c71219e3af21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.391424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f89620a-34f6-4148-82b5-c71219e3af21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:38.393144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f89620a-34f6-4148-82b5-c71219e3af21 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:38.411582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1205/2398] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:38.414130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa90c13-d51d-44e2-af60-dfad922f312f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:38.425543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa90c13-d51d-44e2-af60-dfad922f312f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:38.433945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002a24d-ea08-4050-bc80-ad8142173ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:38.434653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002a24d-ea08-4050-bc80-ad8142173ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.434945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002a24d-ea08-4050-bc80-ad8142173ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.435310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002a24d-ea08-4050-bc80-ad8142173ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.436060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002a24d-ea08-4050-bc80-ad8142173ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:38.438342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e002a24d-ea08-4050-bc80-ad8142173ba9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:38.455205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1194/2399] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:38.459843 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89620a-34f6-4148-82b5-c71219e3af21 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:38.477502 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f89620a-34f6-4148-82b5-c71219e3af21 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:38.485091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ac9b0-b26b-4322-a59b-c6c10efc05d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:38.486262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ac9b0-b26b-4322-a59b-c6c10efc05d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.486549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ac9b0-b26b-4322-a59b-c6c10efc05d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.486830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ac9b0-b26b-4322-a59b-c6c10efc05d4 admin admin] PATH_INFO: `/v3/projects/425c791e08554897bba2334bb83f52d5/users/6e98fb949b374f7c860946d5c72cbd45/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.508380 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ac9b0-b26b-4322-a59b-c6c10efc05d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=425c791e08554897bba2334bb83f52d5, user_id=6e98fb949b374f7c860946d5c72cbd45, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:38.513865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ac9b0-b26b-4322-a59b-c6c10efc05d4 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:38.547095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1206/2400] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:38 2022] PUT /identity/v3/projects/425c791e08554897bba2334bb83f52d5/users/6e98fb949b374f7c860946d5c72cbd45/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:38.552838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aa90d-c967-4466-82e5-e0bef1b1232e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.553144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aa90d-c967-4466-82e5-e0bef1b1232e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.554648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36aa90d-c967-4466-82e5-e0bef1b1232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.601188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b36aa90d-c967-4466-82e5-e0bef1b1232e None None] MFA Rules not processed for user `6e98fb949b374f7c860946d5c72cbd45`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:38.602964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b36aa90d-c967-4466-82e5-e0bef1b1232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:38.694311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776385b-b40b-433c-a563-3240d398f30c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.694627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776385b-b40b-433c-a563-3240d398f30c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.694916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776385b-b40b-433c-a563-3240d398f30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.707973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1195/2401] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:38.712354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aa90d-c967-4466-82e5-e0bef1b1232e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:38.735233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4776385b-b40b-433c-a563-3240d398f30c None None] MFA Rules not processed for user `52315700f2a8435abe22ed8a995d6275`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:38.736824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4776385b-b40b-433c-a563-3240d398f30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:38.764890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:38.787440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:38.789372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36aa90d-c967-4466-82e5-e0bef1b1232e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:38.795823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6e751-983a-4ed5-acba-cc32ed59a640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:38.796915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6e751-983a-4ed5-acba-cc32ed59a640 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.797273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6e751-983a-4ed5-acba-cc32ed59a640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.797638 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6e751-983a-4ed5-acba-cc32ed59a640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.798515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6e751-983a-4ed5-acba-cc32ed59a640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:38.800611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6e751-983a-4ed5-acba-cc32ed59a640 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:38.821129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1207/2402] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:38 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:21:38.835346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1196/2403] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:38 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:21:38.837605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776385b-b40b-433c-a563-3240d398f30c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:38.841335 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6e751-983a-4ed5-acba-cc32ed59a640 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:38.843783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a5e6e751-983a-4ed5-acba-cc32ed59a640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:38.855758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4776385b-b40b-433c-a563-3240d398f30c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:38.929321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6e751-983a-4ed5-acba-cc32ed59a640 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:38.937804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddceb1-3b74-406e-81fc-49d11f82cb23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:38.938177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dac8f0-0ba4-4894-b69a-e07da656e4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:38.938866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dac8f0-0ba4-4894-b69a-e07da656e4ee admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.939191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dac8f0-0ba4-4894-b69a-e07da656e4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.939479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dac8f0-0ba4-4894-b69a-e07da656e4ee admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.939982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddceb1-3b74-406e-81fc-49d11f82cb23 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.940225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dac8f0-0ba4-4894-b69a-e07da656e4ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:38.940939 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddceb1-3b74-406e-81fc-49d11f82cb23 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.941447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddceb1-3b74-406e-81fc-49d11f82cb23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.942204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dac8f0-0ba4-4894-b69a-e07da656e4ee admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:38.943932 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b9ddceb1-3b74-406e-81fc-49d11f82cb23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:38.967014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1208/2404] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:38 2022] POST /identity/v3/users => generated 295 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:21:38.972208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dac8f0-0ba4-4894-b69a-e07da656e4ee admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:38.983944 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dac8f0-0ba4-4894-b69a-e07da656e4ee admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:38.990610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0470ef0-146c-42be-8d3f-796dc09723ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:38.991375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0470ef0-146c-42be-8d3f-796dc09723ae admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:38.992356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0470ef0-146c-42be-8d3f-796dc09723ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:38.992356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0470ef0-146c-42be-8d3f-796dc09723ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:38.992752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0470ef0-146c-42be-8d3f-796dc09723ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:38.994418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0470ef0-146c-42be-8d3f-796dc09723ae admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:39.007796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1209/2405] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:39.011914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0470ef0-146c-42be-8d3f-796dc09723ae admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:39.026082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0470ef0-146c-42be-8d3f-796dc09723ae admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:39.028055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddceb1-3b74-406e-81fc-49d11f82cb23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:39.029724 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddceb1-3b74-406e-81fc-49d11f82cb23 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:39.031352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa12e7-7a71-4efe-aec0-7f3c01944357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:39.032317 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa12e7-7a71-4efe-aec0-7f3c01944357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:39.032678 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa12e7-7a71-4efe-aec0-7f3c01944357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:39.033030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa12e7-7a71-4efe-aec0-7f3c01944357 admin admin] PATH_INFO: `/v3/projects/d8c63f46827c4fc1a5e3beaaf6979416/users/8b311175b5804c62afdbf195a365d9fb/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:39.040843 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1197/2406] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:39.051978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa12e7-7a71-4efe-aec0-7f3c01944357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c63f46827c4fc1a5e3beaaf6979416, user_id=8b311175b5804c62afdbf195a365d9fb, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:39.054796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa12e7-7a71-4efe-aec0-7f3c01944357 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:39.072393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1210/2407] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:39 2022] PUT /identity/v3/projects/d8c63f46827c4fc1a5e3beaaf6979416/users/8b311175b5804c62afdbf195a365d9fb/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:21:39.075970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddceb1-3b74-406e-81fc-49d11f82cb23 service nova] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:39.087457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddceb1-3b74-406e-81fc-49d11f82cb23 service nova] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:39.149961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b290c-8adc-47b2-80f9-1c57979ac742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:39.150707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b290c-8adc-47b2-80f9-1c57979ac742 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:39.150991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b290c-8adc-47b2-80f9-1c57979ac742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:39.151308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b290c-8adc-47b2-80f9-1c57979ac742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:39.152246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b290c-8adc-47b2-80f9-1c57979ac742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:39.153688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b290c-8adc-47b2-80f9-1c57979ac742 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:39.155317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:39.174246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1198/2408] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:39.175685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa12e7-7a71-4efe-aec0-7f3c01944357 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:39.190527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa12e7-7a71-4efe-aec0-7f3c01944357 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:39.209009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592010-11fa-4266-b53a-12e9e6fcc70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:39.209925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592010-11fa-4266-b53a-12e9e6fcc70d admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:39.210195 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592010-11fa-4266-b53a-12e9e6fcc70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:39.211529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07592010-11fa-4266-b53a-12e9e6fcc70d admin admin] PATH_INFO: `/v3/projects/d8c63f46827c4fc1a5e3beaaf6979416/users/8b311175b5804c62afdbf195a365d9fb/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:39.218229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07592010-11fa-4266-b53a-12e9e6fcc70d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8c63f46827c4fc1a5e3beaaf6979416, user_id=8b311175b5804c62afdbf195a365d9fb, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:39.221247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07592010-11fa-4266-b53a-12e9e6fcc70d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:39.249663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1211/2409] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:39 2022] PUT /identity/v3/projects/d8c63f46827c4fc1a5e3beaaf6979416/users/8b311175b5804c62afdbf195a365d9fb/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:21:39.319385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b41afb-6529-41ce-9b54-2d543b7da962 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:39.319889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b41afb-6529-41ce-9b54-2d543b7da962 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:39.319993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b41afb-6529-41ce-9b54-2d543b7da962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:39.350851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a3b41afb-6529-41ce-9b54-2d543b7da962 None None] MFA Rules not processed for user `8b311175b5804c62afdbf195a365d9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:39.352294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a3b41afb-6529-41ce-9b54-2d543b7da962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:39.455029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1199/2410] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:39.467228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592010-11fa-4266-b53a-12e9e6fcc70d admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:39.469505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-07592010-11fa-4266-b53a-12e9e6fcc70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:39.536591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07592010-11fa-4266-b53a-12e9e6fcc70d admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:39.541134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e57295-b8b6-4591-9905-ef2c2cfce402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:39.542253 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e57295-b8b6-4591-9905-ef2c2cfce402 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:39.542558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e57295-b8b6-4591-9905-ef2c2cfce402 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:39.547656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e57295-b8b6-4591-9905-ef2c2cfce402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:39.592421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:39.626880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e57295-b8b6-4591-9905-ef2c2cfce402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:39.628620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e57295-b8b6-4591-9905-ef2c2cfce402 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:39.639776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1212/2411] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:40.187636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae708fec-7a68-4e32-82eb-936f56cd4a7b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:40.188128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae708fec-7a68-4e32-82eb-936f56cd4a7b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:40.188413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae708fec-7a68-4e32-82eb-936f56cd4a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:40.211762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ae708fec-7a68-4e32-82eb-936f56cd4a7b None None] MFA Rules not processed for user `2891dbd54dfb47e4a98d47d2d78b7edf`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:40.213668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ae708fec-7a68-4e32-82eb-936f56cd4a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:40.278127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1200/2412] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:41.307293 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e57295-b8b6-4591-9905-ef2c2cfce402 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:41.309174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-63e57295-b8b6-4591-9905-ef2c2cfce402 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:41.321794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e57295-b8b6-4591-9905-ef2c2cfce402 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:41.326586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07e137-a818-4c22-8412-227e25821dc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:41.327296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07e137-a818-4c22-8412-227e25821dc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:41.327579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07e137-a818-4c22-8412-227e25821dc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:41.327864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07e137-a818-4c22-8412-227e25821dc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:41.392947 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07e137-a818-4c22-8412-227e25821dc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:41.395211 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07e137-a818-4c22-8412-227e25821dc3 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:41.412329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1213/2413] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:41.992367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae708fec-7a68-4e32-82eb-936f56cd4a7b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:42.002146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae708fec-7a68-4e32-82eb-936f56cd4a7b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:42.005925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8debf22-27fc-441f-816f-a4bbdd7d330f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:42.006751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8debf22-27fc-441f-816f-a4bbdd7d330f service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:42.007065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8debf22-27fc-441f-816f-a4bbdd7d330f service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:42.007359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8debf22-27fc-441f-816f-a4bbdd7d330f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:42.027587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8debf22-27fc-441f-816f-a4bbdd7d330f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:42.029837 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8debf22-27fc-441f-816f-a4bbdd7d330f service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:42.054656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1201/2414] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:45.965999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07e137-a818-4c22-8412-227e25821dc3 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:46.032332 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07e137-a818-4c22-8412-227e25821dc3 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:46.037371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:46.038862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:46.039305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:46.039711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] PATH_INFO: `/v3/users/758655f1a1464321869d8044a9dcc263` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:46.041841 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=758655f1a1464321869d8044a9dcc263)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:46.044197 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:46.089048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '758655f1a1464321869d8044a9dcc263'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.089698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '758655f1a1464321869d8044a9dcc263'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.090552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '758655f1a1464321869d8044a9dcc263'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.097597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '758655f1a1464321869d8044a9dcc263'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.105769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '758655f1a1464321869d8044a9dcc263'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.113182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:46.125409 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1214/2415] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:45 2022] DELETE /identity/v3/users/758655f1a1464321869d8044a9dcc263 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:46.129898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685e603-e056-44b1-8b8b-8169c60d0bad None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:46.130404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685e603-e056-44b1-8b8b-8169c60d0bad None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:46.130515 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685e603-e056-44b1-8b8b-8169c60d0bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:46.147296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-5685e603-e056-44b1-8b8b-8169c60d0bad None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:46.148480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5685e603-e056-44b1-8b8b-8169c60d0bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:46.206120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1202/2416] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:46.216261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:46.217679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:46.286789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54b53c8-97a5-40bf-9e4d-4b99b924bc71 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:46.290444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6f41b-c18e-4a82-bc91-c5507a35b1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:46.291112 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6f41b-c18e-4a82-bc91-c5507a35b1cf service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:46.291402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6f41b-c18e-4a82-bc91-c5507a35b1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:46.291680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6f41b-c18e-4a82-bc91-c5507a35b1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:46.300437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6f41b-c18e-4a82-bc91-c5507a35b1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:46.301884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6f41b-c18e-4a82-bc91-c5507a35b1cf service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:46.311749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1215/2417] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:46.599884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685e603-e056-44b1-8b8b-8169c60d0bad None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:46.601336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5685e603-e056-44b1-8b8b-8169c60d0bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:46.615269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685e603-e056-44b1-8b8b-8169c60d0bad None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:46.620703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:46.621580 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:46.621866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:46.622146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] PATH_INFO: `/v3/projects/7fd71b0ea6a24683b4343a6a1a288a8e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:46.623363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fd71b0ea6a24683b4343a6a1a288a8e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:46.625268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:46.633933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:46.672228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fd71b0ea6a24683b4343a6a1a288a8e', 'request_id': 'req-60429104-759b-4faf-a57e-1e76004e52ab'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.679161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fd71b0ea6a24683b4343a6a1a288a8e', 'request_id': 'req-60429104-759b-4faf-a57e-1e76004e52ab'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.685969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fd71b0ea6a24683b4343a6a1a288a8e', 'request_id': 'req-60429104-759b-4faf-a57e-1e76004e52ab'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.696270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] The token cache is being invalidate because project 7fd71b0ea6a24683b4343a6a1a288a8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:46.696566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.697199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.698712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1203/2418] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:46 2022] DELETE /identity/v3/projects/7fd71b0ea6a24683b4343a6a1a288a8e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:46.702729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6f41b-c18e-4a82-bc91-c5507a35b1cf service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:46.712578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6f41b-c18e-4a82-bc91-c5507a35b1cf service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:46.772927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:46.773797 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:46.774083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:46.774362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] PATH_INFO: `/v3/users/6e98fb949b374f7c860946d5c72cbd45` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:46.776055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e98fb949b374f7c860946d5c72cbd45)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:46.777966 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:46.829222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e98fb949b374f7c860946d5c72cbd45'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.829925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e98fb949b374f7c860946d5c72cbd45'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.830594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e98fb949b374f7c860946d5c72cbd45'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.838851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e98fb949b374f7c860946d5c72cbd45'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.846796 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e98fb949b374f7c860946d5c72cbd45'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:46.870584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1216/2419] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:46 2022] DELETE /identity/v3/users/6e98fb949b374f7c860946d5c72cbd45 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:47.166130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:47.167675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:47.180611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60429104-759b-4faf-a57e-1e76004e52ab admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:47.241134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:47.241918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:47.242228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:47.242507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] PATH_INFO: `/v3/projects/425c791e08554897bba2334bb83f52d5` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:47.243782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=425c791e08554897bba2334bb83f52d5)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:47.245745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:47.294785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '425c791e08554897bba2334bb83f52d5', 'request_id': 'req-406d4dcf-3228-4ffd-886e-9abd54037a2e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.301908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '425c791e08554897bba2334bb83f52d5', 'request_id': 'req-406d4dcf-3228-4ffd-886e-9abd54037a2e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.308117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '425c791e08554897bba2334bb83f52d5', 'request_id': 'req-406d4dcf-3228-4ffd-886e-9abd54037a2e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.318118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] The token cache is being invalidate because project 425c791e08554897bba2334bb83f52d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:47.318408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.319030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.320605 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1204/2420] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:47 2022] DELETE /identity/v3/projects/425c791e08554897bba2334bb83f52d5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:47.325051 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:47.326446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:47.339650 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc61e99c-7b12-4753-b12e-af515ff5f51c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:47.347531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:47.397640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:47.398578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:47.398819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:47.399275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] PATH_INFO: `/v3/users/8b311175b5804c62afdbf195a365d9fb` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:47.400371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b311175b5804c62afdbf195a365d9fb)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:47.402305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:47.452589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b311175b5804c62afdbf195a365d9fb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.453258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b311175b5804c62afdbf195a365d9fb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.453880 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b311175b5804c62afdbf195a365d9fb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.461016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b311175b5804c62afdbf195a365d9fb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.469166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-4d1e3bb9-22a9-4207-ac52-f29fa6aacf75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b311175b5804c62afdbf195a365d9fb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.486772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1217/2421] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:47 2022] DELETE /identity/v3/users/8b311175b5804c62afdbf195a365d9fb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:47.766452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:47.767909 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:47.781270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d4dcf-3228-4ffd-886e-9abd54037a2e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:47.806238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:47.840756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:47.841641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:47.841922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:47.842317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] PATH_INFO: `/v3/projects/d8c63f46827c4fc1a5e3beaaf6979416` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:47.843417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8c63f46827c4fc1a5e3beaaf6979416)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:47.845337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:47.890666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8c63f46827c4fc1a5e3beaaf6979416', 'request_id': 'req-90d6b8b1-6efa-4606-bb93-2d96493b8c98'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.897693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8c63f46827c4fc1a5e3beaaf6979416', 'request_id': 'req-90d6b8b1-6efa-4606-bb93-2d96493b8c98'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.904610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8c63f46827c4fc1a5e3beaaf6979416', 'request_id': 'req-90d6b8b1-6efa-4606-bb93-2d96493b8c98'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.914614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] The token cache is being invalidate because project d8c63f46827c4fc1a5e3beaaf6979416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:47.914906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.915564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-90d6b8b1-6efa-4606-bb93-2d96493b8c98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:47.917593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1205/2422] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:47 2022] DELETE /identity/v3/projects/d8c63f46827c4fc1a5e3beaaf6979416 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:47.922547 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f2968-2a2b-44f1-b3d9-ab60fa96414f None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:47.922845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f2968-2a2b-44f1-b3d9-ab60fa96414f None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:47.923166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f2968-2a2b-44f1-b3d9-ab60fa96414f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:47.942009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4a5f2968-2a2b-44f1-b3d9-ab60fa96414f None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:47.943254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4a5f2968-2a2b-44f1-b3d9-ab60fa96414f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:48.008423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1218/2423] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:48.016116 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a994c-25ec-412a-927a-dfadb4090f9a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:48.016363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a994c-25ec-412a-927a-dfadb4090f9a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:48.016641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a994c-25ec-412a-927a-dfadb4090f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:48.034325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ce5a994c-25ec-412a-927a-dfadb4090f9a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:48.035565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ce5a994c-25ec-412a-927a-dfadb4090f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:48.044226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1206/2424] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:48.048351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f2968-2a2b-44f1-b3d9-ab60fa96414f None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:48.058778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f2968-2a2b-44f1-b3d9-ab60fa96414f None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:48.063445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27a0cb-4114-4f24-8531-6cc719445db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:48.064107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27a0cb-4114-4f24-8531-6cc719445db2 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:48.064395 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27a0cb-4114-4f24-8531-6cc719445db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:48.064681 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27a0cb-4114-4f24-8531-6cc719445db2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:48.065625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27a0cb-4114-4f24-8531-6cc719445db2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:21:48.065945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27a0cb-4114-4f24-8531-6cc719445db2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:48.067585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27a0cb-4114-4f24-8531-6cc719445db2 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:48.079979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1219/2425] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:21:48.084022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a994c-25ec-412a-927a-dfadb4090f9a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:48.093464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a994c-25ec-412a-927a-dfadb4090f9a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:48.098144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c124077-9519-4201-80dc-e31f12600512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:48.098829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c124077-9519-4201-80dc-e31f12600512 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:48.099150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c124077-9519-4201-80dc-e31f12600512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:48.099433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c124077-9519-4201-80dc-e31f12600512 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:48.100133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c124077-9519-4201-80dc-e31f12600512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:48.102052 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c124077-9519-4201-80dc-e31f12600512 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:48.128533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1207/2426] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:21:48 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:21:48.132836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27a0cb-4114-4f24-8531-6cc719445db2 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:48.142810 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27a0cb-4114-4f24-8531-6cc719445db2 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:48.155115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:48.205562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff19b5-7979-471d-9bba-021bdb699d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:48.206613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff19b5-7979-471d-9bba-021bdb699d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:48.207012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff19b5-7979-471d-9bba-021bdb699d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:48.207439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff19b5-7979-471d-9bba-021bdb699d2d admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:48.208442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff19b5-7979-471d-9bba-021bdb699d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:48.211300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ff19b5-7979-471d-9bba-021bdb699d2d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:48.245572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1220/2427] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:21:48 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:21:48.250331 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c124077-9519-4201-80dc-e31f12600512 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:48.268946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c124077-9519-4201-80dc-e31f12600512 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:48.277163 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5390c17-fdab-4558-be64-3d6d923c9329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:48.278334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5390c17-fdab-4558-be64-3d6d923c9329 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:48.278849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5390c17-fdab-4558-be64-3d6d923c9329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:48.279385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5390c17-fdab-4558-be64-3d6d923c9329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:48.280767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5390c17-fdab-4558-be64-3d6d923c9329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:48.283945 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5390c17-fdab-4558-be64-3d6d923c9329 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:48.298463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1208/2428] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:21:48.302476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff19b5-7979-471d-9bba-021bdb699d2d admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:48.313292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ff19b5-7979-471d-9bba-021bdb699d2d admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:48.318032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f24143-37e2-4eea-b61b-4d564949fc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:48.318767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f24143-37e2-4eea-b61b-4d564949fc8b admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:48.319083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f24143-37e2-4eea-b61b-4d564949fc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:48.319368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f24143-37e2-4eea-b61b-4d564949fc8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:48.320120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f24143-37e2-4eea-b61b-4d564949fc8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:48.321717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f24143-37e2-4eea-b61b-4d564949fc8b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:48.331835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1221/2429] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:21:48.335993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5390c17-fdab-4558-be64-3d6d923c9329 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:48.346910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5390c17-fdab-4558-be64-3d6d923c9329 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:48.351599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861d707-8217-44df-b844-afbcdd930f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:48.352480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861d707-8217-44df-b844-afbcdd930f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:48.352770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861d707-8217-44df-b844-afbcdd930f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:48.353054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861d707-8217-44df-b844-afbcdd930f49 admin admin] PATH_INFO: `/v3/projects/94b6ab6c23da4b4d88c5c8db88b3d392/users/bd1a8e5299c44421a4daf1d6beb88af9/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:48.368028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861d707-8217-44df-b844-afbcdd930f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b6ab6c23da4b4d88c5c8db88b3d392, user_id=bd1a8e5299c44421a4daf1d6beb88af9, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:48.371237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861d707-8217-44df-b844-afbcdd930f49 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:48.383969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1209/2430] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:21:48 2022] PUT /identity/v3/projects/94b6ab6c23da4b4d88c5c8db88b3d392/users/bd1a8e5299c44421a4daf1d6beb88af9/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:21:48.395857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b8d5e-7736-4fd3-98e9-3d6539ace536 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:48.396150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b8d5e-7736-4fd3-98e9-3d6539ace536 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:48.396430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b8d5e-7736-4fd3-98e9-3d6539ace536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:48.422199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-a20b8d5e-7736-4fd3-98e9-3d6539ace536 None None] MFA Rules not processed for user `bd1a8e5299c44421a4daf1d6beb88af9`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:48.423490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a20b8d5e-7736-4fd3-98e9-3d6539ace536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:48.486746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1222/2431] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:21:48.498544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861d707-8217-44df-b844-afbcdd930f49 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:48.500273 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f861d707-8217-44df-b844-afbcdd930f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:48.562787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861d707-8217-44df-b844-afbcdd930f49 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:48.566391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b72ea-efec-476b-8690-99e226dfa9ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:48.567057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b72ea-efec-476b-8690-99e226dfa9ea service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:48.567359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b72ea-efec-476b-8690-99e226dfa9ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:48.567642 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b72ea-efec-476b-8690-99e226dfa9ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:48.574256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:48.576721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b72ea-efec-476b-8690-99e226dfa9ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:48.578702 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b72ea-efec-476b-8690-99e226dfa9ea service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:48.589324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1210/2432] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:58.483903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=58,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:58.589848 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:21:58.681304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b8d5e-7736-4fd3-98e9-3d6539ace536 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:58.741934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b8d5e-7736-4fd3-98e9-3d6539ace536 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:58.746855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:58.747751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:58.748033 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:58.748307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] PATH_INFO: `/v3/users/bd1a8e5299c44421a4daf1d6beb88af9` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:58.749480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd1a8e5299c44421a4daf1d6beb88af9)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:58.751431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:58.796635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd1a8e5299c44421a4daf1d6beb88af9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:58.797326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd1a8e5299c44421a4daf1d6beb88af9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:58.797922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd1a8e5299c44421a4daf1d6beb88af9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:58.805173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd1a8e5299c44421a4daf1d6beb88af9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:58.812846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd1a8e5299c44421a4daf1d6beb88af9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:58.829773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1223/2433] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:21:58 2022] DELETE /identity/v3/users/bd1a8e5299c44421a4daf1d6beb88af9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:21:58.834257 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bd486-da31-41a8-ab98-677df61c587b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:58.834553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bd486-da31-41a8-ab98-677df61c587b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:58.834839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bd486-da31-41a8-ab98-677df61c587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:58.852756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-627bd486-da31-41a8-ab98-677df61c587b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:21:58.854618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-627bd486-da31-41a8-ab98-677df61c587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:58.911854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1211/2434] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:21:58.921537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:58.922914 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:58.980421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d904ba-828c-42e6-b7b4-dc050039f7c9 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:58.984167 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d4486-ef5f-4dbb-80f3-1d84fca4abe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:58.984811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d4486-ef5f-4dbb-80f3-1d84fca4abe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:58.985103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d4486-ef5f-4dbb-80f3-1d84fca4abe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:58.985385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d4486-ef5f-4dbb-80f3-1d84fca4abe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:58.995008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d4486-ef5f-4dbb-80f3-1d84fca4abe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:58.996430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4d4486-ef5f-4dbb-80f3-1d84fca4abe4 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:59.005779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1224/2435] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:21:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:21:59.275248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bd486-da31-41a8-ab98-677df61c587b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:21:59.276829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-627bd486-da31-41a8-ab98-677df61c587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:21:59.288925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bd486-da31-41a8-ab98-677df61c587b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:21:59.294173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:21:59.295060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:21:59.295492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:21:59.295777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] PATH_INFO: `/v3/projects/94b6ab6c23da4b4d88c5c8db88b3d392` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:21:59.296985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94b6ab6c23da4b4d88c5c8db88b3d392)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:21:59.298948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:21:59.344889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94b6ab6c23da4b4d88c5c8db88b3d392', 'request_id': 'req-ac62a10e-88b0-45ea-b183-7df54b9397a5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:59.352148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94b6ab6c23da4b4d88c5c8db88b3d392', 'request_id': 'req-ac62a10e-88b0-45ea-b183-7df54b9397a5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:59.358692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94b6ab6c23da4b4d88c5c8db88b3d392', 'request_id': 'req-ac62a10e-88b0-45ea-b183-7df54b9397a5'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:59.368907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] The token cache is being invalidate because project 94b6ab6c23da4b4d88c5c8db88b3d392 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:21:59.369842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:59.370486 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:21:59.372063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1212/2436] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:21:59 2022] DELETE /identity/v3/projects/94b6ab6c23da4b4d88c5c8db88b3d392 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:22:09.001588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:22:09.365960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:22:10.406830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d4486-ef5f-4dbb-80f3-1d84fca4abe4 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:22:10.416044 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4d4486-ef5f-4dbb-80f3-1d84fca4abe4 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:22:10.467314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ab2b1-32c3-4538-a03c-f6d81afd3e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:22:10.467976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ab2b1-32c3-4538-a03c-f6d81afd3e9b service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:10.468284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ab2b1-32c3-4538-a03c-f6d81afd3e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:10.468568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ab2b1-32c3-4538-a03c-f6d81afd3e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:10.471201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2f2ab2b1-32c3-4538-a03c-f6d81afd3e9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:10.481656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ab2b1-32c3-4538-a03c-f6d81afd3e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:22:10.483124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ab2b1-32c3-4538-a03c-f6d81afd3e9b service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:22:10.491314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1225/2437] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:22:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:22:11.410172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:22:11.411777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:11.483644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62a10e-88b0-45ea-b183-7df54b9397a5 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:22:11.488633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa418-3a47-41ff-9159-3f551c8a293f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:22:11.489288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa418-3a47-41ff-9159-3f551c8a293f service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:11.489576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa418-3a47-41ff-9159-3f551c8a293f service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:11.489855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa418-3a47-41ff-9159-3f551c8a293f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:11.492051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b45fa418-3a47-41ff-9159-3f551c8a293f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:11.506690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fa418-3a47-41ff-9159-3f551c8a293f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:22:11.508344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fa418-3a47-41ff-9159-3f551c8a293f service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:22:11.518788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1213/2438] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:22:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:22:15.646420 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fee458-5bb4-4d32-87bc-97e2048ccf13 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:15.646900 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fee458-5bb4-4d32-87bc-97e2048ccf13 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:15.647015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fee458-5bb4-4d32-87bc-97e2048ccf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:15.669701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-05fee458-5bb4-4d32-87bc-97e2048ccf13 None None] MFA Rules not processed for user `0187789947064682a8d9e685c2aadb9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:22:15.671141 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-05fee458-5bb4-4d32-87bc-97e2048ccf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:15.731524 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1226/2439] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:22:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:22:21.514485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=32 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:22:25.724149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=51 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:22:30.350542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa418-3a47-41ff-9159-3f551c8a293f service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:22:30.351982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b45fa418-3a47-41ff-9159-3f551c8a293f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:30.363730 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa418-3a47-41ff-9159-3f551c8a293f service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:22:30.367281 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cacfd-9195-40ec-b4cb-961e2b1e7158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:22:30.367911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cacfd-9195-40ec-b4cb-961e2b1e7158 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:30.368195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cacfd-9195-40ec-b4cb-961e2b1e7158 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:30.368469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cacfd-9195-40ec-b4cb-961e2b1e7158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:30.370242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-330cacfd-9195-40ec-b4cb-961e2b1e7158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:30.431132 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cacfd-9195-40ec-b4cb-961e2b1e7158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:22:30.432558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cacfd-9195-40ec-b4cb-961e2b1e7158 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:22:30.442057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1214/2440] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:22:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:22:38.598177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4a206-916a-4e18-8459-437053a88dcf None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:38.598788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4a206-916a-4e18-8459-437053a88dcf None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:38.598911 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4a206-916a-4e18-8459-437053a88dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:38.618950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-e3c4a206-916a-4e18-8459-437053a88dcf None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:22:38.620189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e3c4a206-916a-4e18-8459-437053a88dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:38.675745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1227/2441] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:22:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:22:38.726327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cacfd-9195-40ec-b4cb-961e2b1e7158 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:22:38.727725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-330cacfd-9195-40ec-b4cb-961e2b1e7158 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:38.798582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cacfd-9195-40ec-b4cb-961e2b1e7158 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:22:38.803807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eeeac-3748-46ea-bed0-6719e5dcf2ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:22:38.804454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eeeac-3748-46ea-bed0-6719e5dcf2ae service swift] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:38.804738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eeeac-3748-46ea-bed0-6719e5dcf2ae service swift] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:38.805012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eeeac-3748-46ea-bed0-6719e5dcf2ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:38.813280 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eeeac-3748-46ea-bed0-6719e5dcf2ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:22:38.814885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eeeac-3748-46ea-bed0-6719e5dcf2ae service swift] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:22:38.822729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1215/2442] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:22:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:22:44.965142 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4a206-916a-4e18-8459-437053a88dcf None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:22:44.968228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e3c4a206-916a-4e18-8459-437053a88dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:45.034241 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4a206-916a-4e18-8459-437053a88dcf None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:22:45.037743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a79b5-d4f3-4d24-9f4d-5679d19eb3a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:22:45.038392 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a79b5-d4f3-4d24-9f4d-5679d19eb3a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:45.038677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a79b5-d4f3-4d24-9f4d-5679d19eb3a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:45.038954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a79b5-d4f3-4d24-9f4d-5679d19eb3a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:45.040775 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-379a79b5-d4f3-4d24-9f4d-5679d19eb3a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:45.126064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a79b5-d4f3-4d24-9f4d-5679d19eb3a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:22:45.127605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a79b5-d4f3-4d24-9f4d-5679d19eb3a6 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:22:45.139505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1228/2443] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:22:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:22:48.819590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=62 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:22:55.133910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=73 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:22:56.137890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0540f7-08f8-4ab0-a2f0-3c13ee9110aa None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:56.138420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0540f7-08f8-4ab0-a2f0-3c13ee9110aa None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:56.138527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0540f7-08f8-4ab0-a2f0-3c13ee9110aa None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:56.140043 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1216/2444] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:22:56.145885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381b7be-fa62-4aec-a1ec-e384a0116ad6 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:56.146372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381b7be-fa62-4aec-a1ec-e384a0116ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:56.146613 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381b7be-fa62-4aec-a1ec-e384a0116ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:56.166034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-2381b7be-fa62-4aec-a1ec-e384a0116ad6 None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:22:56.167372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2381b7be-fa62-4aec-a1ec-e384a0116ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:56.246234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1229/2445] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:22:56.258490 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0540f7-08f8-4ab0-a2f0-3c13ee9110aa None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:22:56.260268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9d0540f7-08f8-4ab0-a2f0-3c13ee9110aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:56.319931 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0540f7-08f8-4ab0-a2f0-3c13ee9110aa None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:22:56.323610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d7a118-2cf6-4784-9a47-9319afb15a46 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '450bcd0f22b840c299d08ce9433d6c85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:22:56.324468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7a118-2cf6-4784-9a47-9319afb15a46 service glance] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:56.324758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7a118-2cf6-4784-9a47-9319afb15a46 service glance] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:56.325039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d7a118-2cf6-4784-9a47-9319afb15a46 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:56.335398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7a118-2cf6-4784-9a47-9319afb15a46 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:22:56.336785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d7a118-2cf6-4784-9a47-9319afb15a46 service glance] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:22:56.348525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1217/2446] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:22:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:22:56.415516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b2829-48b3-4cd0-8025-ddba90b6ff73 None None] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:56.415805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b2829-48b3-4cd0-8025-ddba90b6ff73 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:56.416087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b2829-48b3-4cd0-8025-ddba90b6ff73 None None] PATH_INFO: `/` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:56.417487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1230/2447] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:22:56.422760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621858e4-f190-4674-bbe6-2fda042c7f7b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:56.423078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621858e4-f190-4674-bbe6-2fda042c7f7b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:56.423417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621858e4-f190-4674-bbe6-2fda042c7f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:56.441928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-621858e4-f190-4674-bbe6-2fda042c7f7b None None] MFA Rules not processed for user `cd7fa93466c54632ad95834297410e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:22:56.443184 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-621858e4-f190-4674-bbe6-2fda042c7f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:56.454972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1218/2448] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:22:56.467245 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b2829-48b3-4cd0-8025-ddba90b6ff73 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:22:56.478028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b2829-48b3-4cd0-8025-ddba90b6ff73 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:22:56.482121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b82f1a7-19ff-48eb-87d4-9e1e962d58fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:22:56.482746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82f1a7-19ff-48eb-87d4-9e1e962d58fe service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:56.483061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82f1a7-19ff-48eb-87d4-9e1e962d58fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:56.483340 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b82f1a7-19ff-48eb-87d4-9e1e962d58fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:56.490275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82f1a7-19ff-48eb-87d4-9e1e962d58fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:22:56.491748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b82f1a7-19ff-48eb-87d4-9e1e962d58fe service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:22:56.500864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1231/2449] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:22:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:22:56.583339 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b25c-ec72-4abe-afb1-81ef266ed92a None None] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:56.583788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b25c-ec72-4abe-afb1-81ef266ed92a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:56.584071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b25c-ec72-4abe-afb1-81ef266ed92a None None] PATH_INFO: `/` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:56.585513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1219/2450] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 5 15:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 05 15:22:56.590800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec34bd7-d947-4a60-a209-dc3f4d52fd2f None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:56.591127 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec34bd7-d947-4a60-a209-dc3f4d52fd2f None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:56.591419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec34bd7-d947-4a60-a209-dc3f4d52fd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:56.609361 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-2ec34bd7-d947-4a60-a209-dc3f4d52fd2f None None] MFA Rules not processed for user `e7a416f22d33450183801cef2b13822f`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:22:56.610579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2ec34bd7-d947-4a60-a209-dc3f4d52fd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:56.619863 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1232/2451] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 5 15:22:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:22:56.630613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b25c-ec72-4abe-afb1-81ef266ed92a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:22:56.646461 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b25c-ec72-4abe-afb1-81ef266ed92a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:22:56.650207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ebdb1-d60a-4063-9800-3fe5c34a963a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:22:56.650858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ebdb1-d60a-4063-9800-3fe5c34a963a service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:56.651250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ebdb1-d60a-4063-9800-3fe5c34a963a service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:56.651533 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ebdb1-d60a-4063-9800-3fe5c34a963a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:56.663595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ebdb1-d60a-4063-9800-3fe5c34a963a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:22:56.665147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ebdb1-d60a-4063-9800-3fe5c34a963a service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:22:56.676477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1220/2452] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:22:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:22:57.221193 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f179881-abc4-40fe-a70a-76dad6a64bf0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:57.221789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f179881-abc4-40fe-a70a-76dad6a64bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:57.223069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f179881-abc4-40fe-a70a-76dad6a64bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:57.243757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-2f179881-abc4-40fe-a70a-76dad6a64bf0 None None] MFA Rules not processed for user `cc70414d78304179ad171036ae225bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:22:57.244977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2f179881-abc4-40fe-a70a-76dad6a64bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:57.255376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1233/2453] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:22:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:22:58.741647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dae3e-c88f-485a-b460-0a70a4acf85d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:58.742140 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dae3e-c88f-485a-b460-0a70a4acf85d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:58.742236 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dae3e-c88f-485a-b460-0a70a4acf85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:58.760321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-ba1dae3e-c88f-485a-b460-0a70a4acf85d None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:22:58.761512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ba1dae3e-c88f-485a-b460-0a70a4acf85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:22:58.769263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1221/2454] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:22:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:22:58.782070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f179881-abc4-40fe-a70a-76dad6a64bf0 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:22:58.794651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f179881-abc4-40fe-a70a-76dad6a64bf0 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:22:58.799647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef270509-fc29-4ab9-98f2-435dd4f5bfb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:22:58.800315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef270509-fc29-4ab9-98f2-435dd4f5bfb5 service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:22:58.800619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef270509-fc29-4ab9-98f2-435dd4f5bfb5 service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:22:58.800881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef270509-fc29-4ab9-98f2-435dd4f5bfb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:22:58.808789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef270509-fc29-4ab9-98f2-435dd4f5bfb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:22:58.810372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef270509-fc29-4ab9-98f2-435dd4f5bfb5 service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:22:58.818294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1234/2455] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:22:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:23:08.755333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=38 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:23:08.815155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=49 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:23:16.428644 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dae3e-c88f-485a-b460-0a70a4acf85d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:16.438129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dae3e-c88f-485a-b460-0a70a4acf85d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:16.441785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e91d66d-39c0-4e64-87d1-f59b308d0793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:16.442431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91d66d-39c0-4e64-87d1-f59b308d0793 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:16.442717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91d66d-39c0-4e64-87d1-f59b308d0793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:16.442999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e91d66d-39c0-4e64-87d1-f59b308d0793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:16.444812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8e91d66d-39c0-4e64-87d1-f59b308d0793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:16.459085 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91d66d-39c0-4e64-87d1-f59b308d0793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:16.460517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e91d66d-39c0-4e64-87d1-f59b308d0793 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:16.472545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1222/2456] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:23:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:23:17.565635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef270509-fc29-4ab9-98f2-435dd4f5bfb5 service swift] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:17.577143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef270509-fc29-4ab9-98f2-435dd4f5bfb5 service swift] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:17.580957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a469f6-2679-40a4-89b4-60704b857774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:17.581611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a469f6-2679-40a4-89b4-60704b857774 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:17.581899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a469f6-2679-40a4-89b4-60704b857774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:17.582181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a469f6-2679-40a4-89b4-60704b857774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:17.584032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f2a469f6-2679-40a4-89b4-60704b857774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:17.599623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a469f6-2679-40a4-89b4-60704b857774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:17.601132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a469f6-2679-40a4-89b4-60704b857774 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:17.616078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1235/2457] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:23:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:23:23.358982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce5c49-79c0-4441-b07b-886c00cd4c73 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:23.360036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce5c49-79c0-4441-b07b-886c00cd4c73 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:23.360142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce5c49-79c0-4441-b07b-886c00cd4c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:23.377330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8bce5c49-79c0-4441-b07b-886c00cd4c73 None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:23:23.378527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8bce5c49-79c0-4441-b07b-886c00cd4c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:23.385593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1223/2458] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jul 5 15:23:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:23:23.397386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a469f6-2679-40a4-89b4-60704b857774 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:23.409436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a469f6-2679-40a4-89b4-60704b857774 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:23.414298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5db02-c755-4f5f-9173-a4de42002727 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:23.414964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5db02-c755-4f5f-9173-a4de42002727 service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:23.415281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5db02-c755-4f5f-9173-a4de42002727 service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:23.415558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5db02-c755-4f5f-9173-a4de42002727 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:23.424129 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5db02-c755-4f5f-9173-a4de42002727 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:23.425701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5db02-c755-4f5f-9173-a4de42002727 service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:23.433818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1236/2459] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:23:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:23:33.372399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=9 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:23:33.430652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=12 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:23:41.432856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce5c49-79c0-4441-b07b-886c00cd4c73 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:41.442094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce5c49-79c0-4441-b07b-886c00cd4c73 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:41.445591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208e2ca-2e45-4478-9016-0e159bde5b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:41.446227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208e2ca-2e45-4478-9016-0e159bde5b86 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:41.446514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208e2ca-2e45-4478-9016-0e159bde5b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:41.446793 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208e2ca-2e45-4478-9016-0e159bde5b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:41.448666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d208e2ca-2e45-4478-9016-0e159bde5b86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:41.512996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208e2ca-2e45-4478-9016-0e159bde5b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:41.514453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208e2ca-2e45-4478-9016-0e159bde5b86 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:41.524195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1224/2460] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:23:42.252629 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5db02-c755-4f5f-9173-a4de42002727 service swift] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:42.254149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-08c5db02-c755-4f5f-9173-a4de42002727 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:42.266588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5db02-c755-4f5f-9173-a4de42002727 service swift] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:42.270487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab6084-7a69-4726-ae19-a4219a93b634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:42.271567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab6084-7a69-4726-ae19-a4219a93b634 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:42.272094 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab6084-7a69-4726-ae19-a4219a93b634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:42.272506 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab6084-7a69-4726-ae19-a4219a93b634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:42.275063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6dab6084-7a69-4726-ae19-a4219a93b634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:42.343552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab6084-7a69-4726-ae19-a4219a93b634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:42.345000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab6084-7a69-4726-ae19-a4219a93b634 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:42.354642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1237/2461] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:23:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:23:45.211707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208e2ca-2e45-4478-9016-0e159bde5b86 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:45.213336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d208e2ca-2e45-4478-9016-0e159bde5b86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:45.230461 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208e2ca-2e45-4478-9016-0e159bde5b86 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:45.235407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:45.236290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:45.236575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:45.236852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] PATH_INFO: `/v3/users/ef098bbb9ee94bf4aadb4b59516890fc` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:45.252003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef098bbb9ee94bf4aadb4b59516890fc)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:45.254027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:45.301648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef098bbb9ee94bf4aadb4b59516890fc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:23:45.302314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef098bbb9ee94bf4aadb4b59516890fc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:23:45.302914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef098bbb9ee94bf4aadb4b59516890fc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:23:45.311552 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef098bbb9ee94bf4aadb4b59516890fc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:23:45.319014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96ed56d5-3078-4bee-854d-a284e210f037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef098bbb9ee94bf4aadb4b59516890fc'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:23:45.337505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1225/2462] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:23:45 2022] DELETE /identity/v3/users/ef098bbb9ee94bf4aadb4b59516890fc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:23:45.518752 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab6084-7a69-4726-ae19-a4219a93b634 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:45.531959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab6084-7a69-4726-ae19-a4219a93b634 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:45.593995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:45.594865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:45.595186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:45.595458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] PATH_INFO: `/v3/projects/772abc72002b4256b0f600577232889a` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:45.605604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=772abc72002b4256b0f600577232889a)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:45.608066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:45.653965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772abc72002b4256b0f600577232889a', 'request_id': 'req-feec0493-77ed-4947-accd-d81b41993b1a'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:23:45.661266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772abc72002b4256b0f600577232889a', 'request_id': 'req-feec0493-77ed-4947-accd-d81b41993b1a'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:23:45.668280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772abc72002b4256b0f600577232889a', 'request_id': 'req-feec0493-77ed-4947-accd-d81b41993b1a'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:23:45.679355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] The token cache is being invalidate because project 772abc72002b4256b0f600577232889a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:23:45.679643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:23:45.680274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-feec0493-77ed-4947-accd-d81b41993b1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:23:45.681899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1238/2463] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:23:45 2022] DELETE /identity/v3/projects/772abc72002b4256b0f600577232889a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:23:45.687608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f157a5-6e68-4fcf-bde8-b283342e37a6 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:45.687917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f157a5-6e68-4fcf-bde8-b283342e37a6 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:45.688197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f157a5-6e68-4fcf-bde8-b283342e37a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:45.706740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-52f157a5-6e68-4fcf-bde8-b283342e37a6 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:23:45.707963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-52f157a5-6e68-4fcf-bde8-b283342e37a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:45.754576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:23:45.767438 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1226/2464] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:23:45.775407 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e36fa-8907-4bf6-b562-30abc252c1f0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:45.775705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e36fa-8907-4bf6-b562-30abc252c1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:45.775992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e36fa-8907-4bf6-b562-30abc252c1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:45.793898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b06e36fa-8907-4bf6-b562-30abc252c1f0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:23:45.795747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b06e36fa-8907-4bf6-b562-30abc252c1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:45.805023 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1239/2465] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:23:45.809111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f157a5-6e68-4fcf-bde8-b283342e37a6 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:45.818780 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f157a5-6e68-4fcf-bde8-b283342e37a6 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:45.823425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009dfe5-eaaa-4318-adf0-aecf1865f592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:45.824080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009dfe5-eaaa-4318-adf0-aecf1865f592 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:45.824373 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009dfe5-eaaa-4318-adf0-aecf1865f592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:45.824655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009dfe5-eaaa-4318-adf0-aecf1865f592 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:45.825472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009dfe5-eaaa-4318-adf0-aecf1865f592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:23:45.825806 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009dfe5-eaaa-4318-adf0-aecf1865f592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:45.827537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009dfe5-eaaa-4318-adf0-aecf1865f592 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:45.839061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1227/2466] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:23:45.843930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e36fa-8907-4bf6-b562-30abc252c1f0 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:45.853120 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e36fa-8907-4bf6-b562-30abc252c1f0 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:45.858285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5590e-260d-4a47-ae44-b0e1e0750485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:45.858922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5590e-260d-4a47-ae44-b0e1e0750485 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:45.859237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5590e-260d-4a47-ae44-b0e1e0750485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:45.859517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5590e-260d-4a47-ae44-b0e1e0750485 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:45.860210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5590e-260d-4a47-ae44-b0e1e0750485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:45.862417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5590e-260d-4a47-ae44-b0e1e0750485 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:45.866981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:23:45.893102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1240/2467] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:23:45 2022] POST /identity/v3/projects => generated 357 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:23:45.894112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009dfe5-eaaa-4318-adf0-aecf1865f592 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:45.904213 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009dfe5-eaaa-4318-adf0-aecf1865f592 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:45.962445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16e4a1-1787-44aa-80e4-90fb0c564fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:45.962786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16e4a1-1787-44aa-80e4-90fb0c564fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:45.963103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16e4a1-1787-44aa-80e4-90fb0c564fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:45.963383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16e4a1-1787-44aa-80e4-90fb0c564fe1 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:45.964115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16e4a1-1787-44aa-80e4-90fb0c564fe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:45.966306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae16e4a1-1787-44aa-80e4-90fb0c564fe1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:45.987249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1228/2468] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:23:45 2022] POST /identity/v3/users => generated 279 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:23:45.991284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5590e-260d-4a47-ae44-b0e1e0750485 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.001635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5590e-260d-4a47-ae44-b0e1e0750485 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.006378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef5bd3-b2f3-41cf-92a0-655941ea0fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.007074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef5bd3-b2f3-41cf-92a0-655941ea0fef admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.007369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef5bd3-b2f3-41cf-92a0-655941ea0fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.007652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef5bd3-b2f3-41cf-92a0-655941ea0fef admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.008387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef5bd3-b2f3-41cf-92a0-655941ea0fef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.009998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef5bd3-b2f3-41cf-92a0-655941ea0fef admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.020673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1241/2469] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:23:46.024658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16e4a1-1787-44aa-80e4-90fb0c564fe1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.035163 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae16e4a1-1787-44aa-80e4-90fb0c564fe1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.039623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad649e-7656-4be2-9325-3e98e2669cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.040277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad649e-7656-4be2-9325-3e98e2669cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.040611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad649e-7656-4be2-9325-3e98e2669cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.040869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad649e-7656-4be2-9325-3e98e2669cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.041591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad649e-7656-4be2-9325-3e98e2669cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.043224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad649e-7656-4be2-9325-3e98e2669cc7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.054351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1229/2470] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:23:46.058465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef5bd3-b2f3-41cf-92a0-655941ea0fef admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.068548 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef5bd3-b2f3-41cf-92a0-655941ea0fef admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.073214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d773eea-7966-48d6-a080-59b4789b2387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.074089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d773eea-7966-48d6-a080-59b4789b2387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.074376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d773eea-7966-48d6-a080-59b4789b2387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.074662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d773eea-7966-48d6-a080-59b4789b2387 admin admin] PATH_INFO: `/v3/projects/690d313110ea491ba1bd1e910a5041b1/users/ab070ff17c26458ea95a12c7d6c303f4/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.089037 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d773eea-7966-48d6-a080-59b4789b2387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=690d313110ea491ba1bd1e910a5041b1, user_id=ab070ff17c26458ea95a12c7d6c303f4, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.092800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d773eea-7966-48d6-a080-59b4789b2387 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.105055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1242/2471] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:23:46 2022] PUT /identity/v3/projects/690d313110ea491ba1bd1e910a5041b1/users/ab070ff17c26458ea95a12c7d6c303f4/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:23:46.115739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ca3ac-ac08-4231-8b36-66133f37eba8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.116035 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ca3ac-ac08-4231-8b36-66133f37eba8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.116317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ca3ac-ac08-4231-8b36-66133f37eba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.142082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-be7ca3ac-ac08-4231-8b36-66133f37eba8 None None] MFA Rules not processed for user `ab070ff17c26458ea95a12c7d6c303f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:23:46.143304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-be7ca3ac-ac08-4231-8b36-66133f37eba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:46.205399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1230/2472] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:23:46 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:23:46.209903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d773eea-7966-48d6-a080-59b4789b2387 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.272952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d773eea-7966-48d6-a080-59b4789b2387 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.277489 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a591459-2909-4538-a055-6739cd277227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.278148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a591459-2909-4538-a055-6739cd277227 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.278440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a591459-2909-4538-a055-6739cd277227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.278727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a591459-2909-4538-a055-6739cd277227 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.280115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a591459-2909-4538-a055-6739cd277227 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.282043 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a591459-2909-4538-a055-6739cd277227 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.307054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1243/2473] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:23:46 2022] POST /identity/v3/projects => generated 355 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:23:46.311025 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ca3ac-ac08-4231-8b36-66133f37eba8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.322399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ca3ac-ac08-4231-8b36-66133f37eba8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.387495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:23:46.401188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6162e-fdc9-4d7e-8463-526638c61ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.401908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6162e-fdc9-4d7e-8463-526638c61ffc admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.402199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6162e-fdc9-4d7e-8463-526638c61ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.403630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6162e-fdc9-4d7e-8463-526638c61ffc admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.404749 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6162e-fdc9-4d7e-8463-526638c61ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.407879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6162e-fdc9-4d7e-8463-526638c61ffc admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.430335 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1231/2474] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:23:46 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:23:46.434371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a591459-2909-4538-a055-6739cd277227 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.445744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a591459-2909-4538-a055-6739cd277227 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.450486 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9201-c4f3-4320-be83-0e490fe0d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.451200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9201-c4f3-4320-be83-0e490fe0d6ac admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.451494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9201-c4f3-4320-be83-0e490fe0d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.451776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d9201-c4f3-4320-be83-0e490fe0d6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.452509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d9201-c4f3-4320-be83-0e490fe0d6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.454126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0d9201-c4f3-4320-be83-0e490fe0d6ac admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.464749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1244/2475] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:23:46.469160 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6162e-fdc9-4d7e-8463-526638c61ffc admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.479362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6162e-fdc9-4d7e-8463-526638c61ffc admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.484687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66895b01-6059-4626-9200-b9c34015d999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.485565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66895b01-6059-4626-9200-b9c34015d999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.485855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66895b01-6059-4626-9200-b9c34015d999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.486139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66895b01-6059-4626-9200-b9c34015d999 admin admin] PATH_INFO: `/v3/projects/777dca05ac41465bb625a785cbcd6d87/users/c5c6b013add943d8bd4e3c809c7e49f6/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.503922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66895b01-6059-4626-9200-b9c34015d999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=777dca05ac41465bb625a785cbcd6d87, user_id=c5c6b013add943d8bd4e3c809c7e49f6, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.506768 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66895b01-6059-4626-9200-b9c34015d999 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.519806 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1232/2476] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:23:46 2022] PUT /identity/v3/projects/777dca05ac41465bb625a785cbcd6d87/users/c5c6b013add943d8bd4e3c809c7e49f6/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:23:46.523786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9201-c4f3-4320-be83-0e490fe0d6ac admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.534632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0d9201-c4f3-4320-be83-0e490fe0d6ac admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.590084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20892a88-b400-4bba-9cc4-deb132cbe9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.591001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20892a88-b400-4bba-9cc4-deb132cbe9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.591319 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20892a88-b400-4bba-9cc4-deb132cbe9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.591598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20892a88-b400-4bba-9cc4-deb132cbe9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.592320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20892a88-b400-4bba-9cc4-deb132cbe9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.593923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20892a88-b400-4bba-9cc4-deb132cbe9a6 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.603670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1245/2477] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:23:46.608602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66895b01-6059-4626-9200-b9c34015d999 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.618641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66895b01-6059-4626-9200-b9c34015d999 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.623819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd46e65-06c9-463a-bc55-73f84c7a84d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.624690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd46e65-06c9-463a-bc55-73f84c7a84d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.624974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd46e65-06c9-463a-bc55-73f84c7a84d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.625251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd46e65-06c9-463a-bc55-73f84c7a84d7 admin admin] PATH_INFO: `/v3/projects/777dca05ac41465bb625a785cbcd6d87/users/c5c6b013add943d8bd4e3c809c7e49f6/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.627294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd46e65-06c9-463a-bc55-73f84c7a84d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=777dca05ac41465bb625a785cbcd6d87, user_id=c5c6b013add943d8bd4e3c809c7e49f6, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.629912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd46e65-06c9-463a-bc55-73f84c7a84d7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.640489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1233/2478] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:23:46 2022] PUT /identity/v3/projects/777dca05ac41465bb625a785cbcd6d87/users/c5c6b013add943d8bd4e3c809c7e49f6/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:23:46.651418 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a5141-deff-4f6b-ad2d-9a9d6bf8bd66 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.651717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a5141-deff-4f6b-ad2d-9a9d6bf8bd66 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.652006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a5141-deff-4f6b-ad2d-9a9d6bf8bd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.679958 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-aa0a5141-deff-4f6b-ad2d-9a9d6bf8bd66 None None] MFA Rules not processed for user `c5c6b013add943d8bd4e3c809c7e49f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:23:46.681180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-aa0a5141-deff-4f6b-ad2d-9a9d6bf8bd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:46.690485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:23:46.750531 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1246/2479] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:23:46 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:23:46.761664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd46e65-06c9-463a-bc55-73f84c7a84d7 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.763489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3cd46e65-06c9-463a-bc55-73f84c7a84d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:23:46.820815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd46e65-06c9-463a-bc55-73f84c7a84d7 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.824422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b0d21-0510-4a26-be28-963b900d5604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.825060 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b0d21-0510-4a26-be28-963b900d5604 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.825350 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b0d21-0510-4a26-be28-963b900d5604 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.825630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b0d21-0510-4a26-be28-963b900d5604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.835201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b0d21-0510-4a26-be28-963b900d5604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.836645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677b0d21-0510-4a26-be28-963b900d5604 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.847142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1234/2480] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:23:46 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:23:46.918061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a5141-deff-4f6b-ad2d-9a9d6bf8bd66 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:23:46.928280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a5141-deff-4f6b-ad2d-9a9d6bf8bd66 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:23:46.931959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa26d50-2028-4823-8b15-e968a28e24c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:23:46.932604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26d50-2028-4823-8b15-e968a28e24c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:23:46.932893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26d50-2028-4823-8b15-e968a28e24c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:23:46.933175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa26d50-2028-4823-8b15-e968a28e24c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:23:46.988889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa26d50-2028-4823-8b15-e968a28e24c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:23:46.990337 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa26d50-2028-4823-8b15-e968a28e24c5 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:23:46.999776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1247/2481] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:23:46 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:23:56.842832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=47,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:23:56.995121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=52 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:01.609238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b0d21-0510-4a26-be28-963b900d5604 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:01.670402 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b0d21-0510-4a26-be28-963b900d5604 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:01.675534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:01.676432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:01.676715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:01.676993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] PATH_INFO: `/v3/users/ab070ff17c26458ea95a12c7d6c303f4` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:01.678312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab070ff17c26458ea95a12c7d6c303f4)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:01.680318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:01.726354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab070ff17c26458ea95a12c7d6c303f4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:01.727003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab070ff17c26458ea95a12c7d6c303f4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:01.727682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab070ff17c26458ea95a12c7d6c303f4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:01.735815 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab070ff17c26458ea95a12c7d6c303f4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:01.742844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab070ff17c26458ea95a12c7d6c303f4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:01.759978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1235/2482] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:01 2022] DELETE /identity/v3/users/ab070ff17c26458ea95a12c7d6c303f4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:24:01.764537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7875f-242b-4aad-afd8-fd800f3cc5bd None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:01.765035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7875f-242b-4aad-afd8-fd800f3cc5bd None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:01.765137 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7875f-242b-4aad-afd8-fd800f3cc5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:01.782918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-53f7875f-242b-4aad-afd8-fd800f3cc5bd None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:01.784139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-53f7875f-242b-4aad-afd8-fd800f3cc5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:01.849919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1248/2483] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:24:01.860017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:01.861452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:01.917740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5512d-d4f8-4ea9-b838-9f30a03bcb99 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:01.921433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afecc69-ff3b-4e47-acbc-e62f11dcb03c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:01.922104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afecc69-ff3b-4e47-acbc-e62f11dcb03c service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:01.922385 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afecc69-ff3b-4e47-acbc-e62f11dcb03c service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:01.922688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afecc69-ff3b-4e47-acbc-e62f11dcb03c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:01.931835 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afecc69-ff3b-4e47-acbc-e62f11dcb03c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:01.933341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afecc69-ff3b-4e47-acbc-e62f11dcb03c service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:01.942852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1236/2484] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:02.217034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7875f-242b-4aad-afd8-fd800f3cc5bd None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:02.219098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-53f7875f-242b-4aad-afd8-fd800f3cc5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:02.231374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7875f-242b-4aad-afd8-fd800f3cc5bd None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:02.236047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:02.236915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:02.237206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:02.237494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] PATH_INFO: `/v3/projects/690d313110ea491ba1bd1e910a5041b1` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:02.238725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=690d313110ea491ba1bd1e910a5041b1)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:02.240706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:02.287624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690d313110ea491ba1bd1e910a5041b1', 'request_id': 'req-9d0d78fa-c1b1-403f-90df-581a0db94cbe'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.294972 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690d313110ea491ba1bd1e910a5041b1', 'request_id': 'req-9d0d78fa-c1b1-403f-90df-581a0db94cbe'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.301133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690d313110ea491ba1bd1e910a5041b1', 'request_id': 'req-9d0d78fa-c1b1-403f-90df-581a0db94cbe'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.311248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] The token cache is being invalidate because project 690d313110ea491ba1bd1e910a5041b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:02.312048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.312698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.314236 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1249/2485] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:02 2022] DELETE /identity/v3/projects/690d313110ea491ba1bd1e910a5041b1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:02.318237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afecc69-ff3b-4e47-acbc-e62f11dcb03c service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:02.328453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afecc69-ff3b-4e47-acbc-e62f11dcb03c service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:02.346371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:02.386000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:02.386862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:02.387174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:02.387453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] PATH_INFO: `/v3/users/c5c6b013add943d8bd4e3c809c7e49f6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:02.388613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5c6b013add943d8bd4e3c809c7e49f6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:02.391217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:02.433862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5c6b013add943d8bd4e3c809c7e49f6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.434509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5c6b013add943d8bd4e3c809c7e49f6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.435143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5c6b013add943d8bd4e3c809c7e49f6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.442648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5c6b013add943d8bd4e3c809c7e49f6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.450822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ca24aafe-a6a4-4dfb-8b4e-ef6d4d5b11da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5c6b013add943d8bd4e3c809c7e49f6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.468587 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1237/2486] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:02 2022] DELETE /identity/v3/users/c5c6b013add943d8bd4e3c809c7e49f6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:02.737604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:02.739059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:02.752462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d78fa-c1b1-403f-90df-581a0db94cbe admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:02.809381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:02.810302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:02.810537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:02.810815 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] PATH_INFO: `/v3/projects/777dca05ac41465bb625a785cbcd6d87` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:02.812029 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=777dca05ac41465bb625a785cbcd6d87)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:02.813932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:02.845151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:02.864918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '777dca05ac41465bb625a785cbcd6d87', 'request_id': 'req-3e7ca4d4-b848-4f7f-836a-9126103e0443'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.872747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '777dca05ac41465bb625a785cbcd6d87', 'request_id': 'req-3e7ca4d4-b848-4f7f-836a-9126103e0443'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.879460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '777dca05ac41465bb625a785cbcd6d87', 'request_id': 'req-3e7ca4d4-b848-4f7f-836a-9126103e0443'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.891111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] The token cache is being invalidate because project 777dca05ac41465bb625a785cbcd6d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:02.891409 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.892053 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-3e7ca4d4-b848-4f7f-836a-9126103e0443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:02.893825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1250/2487] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:02 2022] DELETE /identity/v3/projects/777dca05ac41465bb625a785cbcd6d87 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:02.899215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb3f09-8aa5-416f-a26a-98ebe7b1d318 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:02.899215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb3f09-8aa5-416f-a26a-98ebe7b1d318 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:02.899795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb3f09-8aa5-416f-a26a-98ebe7b1d318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:02.918098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-32eb3f09-8aa5-416f-a26a-98ebe7b1d318 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:02.919328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-32eb3f09-8aa5-416f-a26a-98ebe7b1d318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:02.978311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1238/2488] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:02.986528 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd015e-dee3-42cf-88f8-70b7cc84093e None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:02.986827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd015e-dee3-42cf-88f8-70b7cc84093e None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:02.987150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd015e-dee3-42cf-88f8-70b7cc84093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.006152 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-32dd015e-dee3-42cf-88f8-70b7cc84093e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:03.007410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-32dd015e-dee3-42cf-88f8-70b7cc84093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:03.015414 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1251/2489] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:03.019526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb3f09-8aa5-416f-a26a-98ebe7b1d318 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.029050 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb3f09-8aa5-416f-a26a-98ebe7b1d318 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.034131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136360-a75c-482b-95bb-21ad593b3324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.034787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136360-a75c-482b-95bb-21ad593b3324 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.035104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136360-a75c-482b-95bb-21ad593b3324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.035391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7136360-a75c-482b-95bb-21ad593b3324 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.036129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7136360-a75c-482b-95bb-21ad593b3324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:24:03.036443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7136360-a75c-482b-95bb-21ad593b3324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.038041 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7136360-a75c-482b-95bb-21ad593b3324 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.054888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1239/2490] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:24:03.056326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd015e-dee3-42cf-88f8-70b7cc84093e None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.065996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd015e-dee3-42cf-88f8-70b7cc84093e None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.070679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfd0fc-f1a5-4edd-8137-138107e0288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.071360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfd0fc-f1a5-4edd-8137-138107e0288e admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.071646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfd0fc-f1a5-4edd-8137-138107e0288e admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.071923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfd0fc-f1a5-4edd-8137-138107e0288e admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.072607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfd0fc-f1a5-4edd-8137-138107e0288e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.074504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfd0fc-f1a5-4edd-8137-138107e0288e admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.100216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1252/2491] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:03 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:03.104309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136360-a75c-482b-95bb-21ad593b3324 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.115214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7136360-a75c-482b-95bb-21ad593b3324 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.141733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:03.174896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cb7f6-9728-420f-8127-65b2404b5634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.175585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cb7f6-9728-420f-8127-65b2404b5634 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.175896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cb7f6-9728-420f-8127-65b2404b5634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.176173 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1cb7f6-9728-420f-8127-65b2404b5634 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.176856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cb7f6-9728-420f-8127-65b2404b5634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.178771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1cb7f6-9728-420f-8127-65b2404b5634 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.199608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1240/2492] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:03 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:03.204330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfd0fc-f1a5-4edd-8137-138107e0288e admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.214834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfd0fc-f1a5-4edd-8137-138107e0288e admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.219884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2c1b2-2e72-4623-8593-b0ae4e6003f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.220549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2c1b2-2e72-4623-8593-b0ae4e6003f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.220840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2c1b2-2e72-4623-8593-b0ae4e6003f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.221121 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2c1b2-2e72-4623-8593-b0ae4e6003f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.221851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2c1b2-2e72-4623-8593-b0ae4e6003f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.223494 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2c1b2-2e72-4623-8593-b0ae4e6003f3 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.233897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1253/2493] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:03.237863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cb7f6-9728-420f-8127-65b2404b5634 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.247773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1cb7f6-9728-420f-8127-65b2404b5634 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.252893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac8e17-26ed-47dc-96e6-dcc963576cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.253551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8e17-26ed-47dc-96e6-dcc963576cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.253841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8e17-26ed-47dc-96e6-dcc963576cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.254122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8e17-26ed-47dc-96e6-dcc963576cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.254893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac8e17-26ed-47dc-96e6-dcc963576cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.256558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac8e17-26ed-47dc-96e6-dcc963576cc4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.267452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1241/2494] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:03.271326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2c1b2-2e72-4623-8593-b0ae4e6003f3 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.281312 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2c1b2-2e72-4623-8593-b0ae4e6003f3 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.286029 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610834c1-cf4f-4ba4-b553-dc226821e4bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.286909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610834c1-cf4f-4ba4-b553-dc226821e4bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.287232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610834c1-cf4f-4ba4-b553-dc226821e4bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.287519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610834c1-cf4f-4ba4-b553-dc226821e4bd admin admin] PATH_INFO: `/v3/projects/34bfccfb786040ca9641fc37f7e78db2/users/6a3c45fa450a40acbd754eb7df9f9ee0/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.302059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610834c1-cf4f-4ba4-b553-dc226821e4bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=34bfccfb786040ca9641fc37f7e78db2, user_id=6a3c45fa450a40acbd754eb7df9f9ee0, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.304777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610834c1-cf4f-4ba4-b553-dc226821e4bd admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.317904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1254/2495] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:03 2022] PUT /identity/v3/projects/34bfccfb786040ca9641fc37f7e78db2/users/6a3c45fa450a40acbd754eb7df9f9ee0/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:03.330045 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc207221-03fa-40ea-b402-abea40ffbe97 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.330340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc207221-03fa-40ea-b402-abea40ffbe97 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.330623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc207221-03fa-40ea-b402-abea40ffbe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.357262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fc207221-03fa-40ea-b402-abea40ffbe97 None None] MFA Rules not processed for user `6a3c45fa450a40acbd754eb7df9f9ee0`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:03.359068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fc207221-03fa-40ea-b402-abea40ffbe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:03.422472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1242/2496] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:03.426858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610834c1-cf4f-4ba4-b553-dc226821e4bd admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.488965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610834c1-cf4f-4ba4-b553-dc226821e4bd admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.493649 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491d336-93a9-4123-ad16-f867c9c22ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.494305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491d336-93a9-4123-ad16-f867c9c22ead admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.494592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491d336-93a9-4123-ad16-f867c9c22ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.494871 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491d336-93a9-4123-ad16-f867c9c22ead admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.495610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491d336-93a9-4123-ad16-f867c9c22ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.497523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491d336-93a9-4123-ad16-f867c9c22ead admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.524844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1255/2497] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:03 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:03.527635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc207221-03fa-40ea-b402-abea40ffbe97 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.538850 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc207221-03fa-40ea-b402-abea40ffbe97 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.592751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924624fd-7e42-43d7-ade1-4fe65eb7a2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.594017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924624fd-7e42-43d7-ade1-4fe65eb7a2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.594239 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924624fd-7e42-43d7-ade1-4fe65eb7a2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.594560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924624fd-7e42-43d7-ade1-4fe65eb7a2d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.595270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924624fd-7e42-43d7-ade1-4fe65eb7a2d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.597198 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924624fd-7e42-43d7-ade1-4fe65eb7a2d3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.619682 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1243/2498] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:03 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:03.623610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491d336-93a9-4123-ad16-f867c9c22ead admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.633813 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491d336-93a9-4123-ad16-f867c9c22ead admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.638426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6e390-774c-4db0-a748-66f4eb6ed7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.639110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6e390-774c-4db0-a748-66f4eb6ed7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.639400 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6e390-774c-4db0-a748-66f4eb6ed7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.640195 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6e390-774c-4db0-a748-66f4eb6ed7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.640912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6e390-774c-4db0-a748-66f4eb6ed7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.642536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc6e390-774c-4db0-a748-66f4eb6ed7c9 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.652870 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1256/2499] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:03.657419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924624fd-7e42-43d7-ade1-4fe65eb7a2d3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.667312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924624fd-7e42-43d7-ade1-4fe65eb7a2d3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.672420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e6883-92a4-4748-a25f-edf320b542cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.673296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e6883-92a4-4748-a25f-edf320b542cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.673584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e6883-92a4-4748-a25f-edf320b542cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.673864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e6883-92a4-4748-a25f-edf320b542cd admin admin] PATH_INFO: `/v3/projects/d7311482a4ad4af79ee0052ae9fbfec0/users/68bad7a1dd904461a9e0698751a739b7/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.689059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e6883-92a4-4748-a25f-edf320b542cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7311482a4ad4af79ee0052ae9fbfec0, user_id=68bad7a1dd904461a9e0698751a739b7, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.691845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e6883-92a4-4748-a25f-edf320b542cd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.704968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1244/2500] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:03 2022] PUT /identity/v3/projects/d7311482a4ad4af79ee0052ae9fbfec0/users/68bad7a1dd904461a9e0698751a739b7/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:03.708899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6e390-774c-4db0-a748-66f4eb6ed7c9 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.719151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6e390-774c-4db0-a748-66f4eb6ed7c9 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.739002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:03.777602 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1273a6cd-1814-4bd8-9523-3c90636ba1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.778305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273a6cd-1814-4bd8-9523-3c90636ba1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.778615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273a6cd-1814-4bd8-9523-3c90636ba1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.778893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1273a6cd-1814-4bd8-9523-3c90636ba1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.779661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273a6cd-1814-4bd8-9523-3c90636ba1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.781867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1273a6cd-1814-4bd8-9523-3c90636ba1d4 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.791844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1257/2501] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:03.796486 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e6883-92a4-4748-a25f-edf320b542cd admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.801863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:03.808002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e6883-92a4-4748-a25f-edf320b542cd admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:03.812829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b8223-b48a-4070-b76c-7d2356de8e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:03.813706 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b8223-b48a-4070-b76c-7d2356de8e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.813992 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b8223-b48a-4070-b76c-7d2356de8e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.814272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778b8223-b48a-4070-b76c-7d2356de8e8c admin admin] PATH_INFO: `/v3/projects/d7311482a4ad4af79ee0052ae9fbfec0/users/68bad7a1dd904461a9e0698751a739b7/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.816254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b8223-b48a-4070-b76c-7d2356de8e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7311482a4ad4af79ee0052ae9fbfec0, user_id=68bad7a1dd904461a9e0698751a739b7, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:03.818986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778b8223-b48a-4070-b76c-7d2356de8e8c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:03.828971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1245/2502] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:03 2022] PUT /identity/v3/projects/d7311482a4ad4af79ee0052ae9fbfec0/users/68bad7a1dd904461a9e0698751a739b7/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:03.838708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26af43-77bc-4051-867d-ce06aa28bbdf None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:03.839194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26af43-77bc-4051-867d-ce06aa28bbdf None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:03.839289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26af43-77bc-4051-867d-ce06aa28bbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:03.865855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-4c26af43-77bc-4051-867d-ce06aa28bbdf None None] MFA Rules not processed for user `68bad7a1dd904461a9e0698751a739b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:03.866983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4c26af43-77bc-4051-867d-ce06aa28bbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:03.937410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1258/2503] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:03.949299 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b8223-b48a-4070-b76c-7d2356de8e8c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:03.951241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-778b8223-b48a-4070-b76c-7d2356de8e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:04.007905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778b8223-b48a-4070-b76c-7d2356de8e8c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:04.011518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6c5d1-4249-4dd6-8871-c79b732f5a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:04.012154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6c5d1-4249-4dd6-8871-c79b732f5a96 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:04.012439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6c5d1-4249-4dd6-8871-c79b732f5a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:04.012718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6c5d1-4249-4dd6-8871-c79b732f5a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:04.021482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6c5d1-4249-4dd6-8871-c79b732f5a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:04.022907 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6c5d1-4249-4dd6-8871-c79b732f5a96 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:04.032648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1246/2504] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:04.112243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26af43-77bc-4051-867d-ce06aa28bbdf None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:04.121689 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26af43-77bc-4051-867d-ce06aa28bbdf None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:04.125237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f990f8-b3e5-4763-a007-2089beaf330f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:04.125868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f990f8-b3e5-4763-a007-2089beaf330f service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:04.126153 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f990f8-b3e5-4763-a007-2089beaf330f service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:04.126430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f990f8-b3e5-4763-a007-2089beaf330f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:04.180837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f990f8-b3e5-4763-a007-2089beaf330f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:04.182268 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f990f8-b3e5-4763-a007-2089beaf330f service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:04.191432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1259/2505] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:08.806831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6c5d1-4249-4dd6-8871-c79b732f5a96 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:08.869582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6c5d1-4249-4dd6-8871-c79b732f5a96 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:08.874231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:08.875834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:08.876136 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:08.876413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] PATH_INFO: `/v3/users/6a3c45fa450a40acbd754eb7df9f9ee0` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:08.877594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3c45fa450a40acbd754eb7df9f9ee0)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:08.879554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:08.923408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3c45fa450a40acbd754eb7df9f9ee0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:08.923994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3c45fa450a40acbd754eb7df9f9ee0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:08.924602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3c45fa450a40acbd754eb7df9f9ee0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:08.931853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3c45fa450a40acbd754eb7df9f9ee0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:08.938693 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3c45fa450a40acbd754eb7df9f9ee0'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:08.956005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1247/2506] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:08 2022] DELETE /identity/v3/users/6a3c45fa450a40acbd754eb7df9f9ee0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:08.960630 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b799-cb54-4920-9a3d-eb4034cb263b None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:08.961085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b799-cb54-4920-9a3d-eb4034cb263b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:08.961226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e728b799-cb54-4920-9a3d-eb4034cb263b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:08.978846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-e728b799-cb54-4920-9a3d-eb4034cb263b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:08.980107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e728b799-cb54-4920-9a3d-eb4034cb263b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:09.019801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:09.038384 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1260/2507] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:09.047933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:09.049330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:09.063709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:09.111830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52db20-fa55-476c-9d13-9e56961b011c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:09.115578 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c76d-cd6a-4463-af59-7fc0ca57832f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:09.116226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c76d-cd6a-4463-af59-7fc0ca57832f service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:09.116512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c76d-cd6a-4463-af59-7fc0ca57832f service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:09.116788 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5c76d-cd6a-4463-af59-7fc0ca57832f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:09.126291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5c76d-cd6a-4463-af59-7fc0ca57832f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:09.127763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd5c76d-cd6a-4463-af59-7fc0ca57832f service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:09.137180 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1248/2508] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:09.398869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728b799-cb54-4920-9a3d-eb4034cb263b None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:09.400828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e728b799-cb54-4920-9a3d-eb4034cb263b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:09.424883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e728b799-cb54-4920-9a3d-eb4034cb263b None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:09.432784 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:09.433598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:09.433945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:09.434264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] PATH_INFO: `/v3/projects/34bfccfb786040ca9641fc37f7e78db2` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:09.435955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34bfccfb786040ca9641fc37f7e78db2)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:09.437658 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:09.486800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34bfccfb786040ca9641fc37f7e78db2', 'request_id': 'req-abc5bd17-a309-4a35-b1a2-1edb21241350'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:09.496117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34bfccfb786040ca9641fc37f7e78db2', 'request_id': 'req-abc5bd17-a309-4a35-b1a2-1edb21241350'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:09.503948 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34bfccfb786040ca9641fc37f7e78db2', 'request_id': 'req-abc5bd17-a309-4a35-b1a2-1edb21241350'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:09.514719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] The token cache is being invalidate because project 34bfccfb786040ca9641fc37f7e78db2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:09.515010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:09.515678 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:09.517239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1261/2509] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:09 2022] DELETE /identity/v3/projects/34bfccfb786040ca9641fc37f7e78db2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:09.521423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c76d-cd6a-4463-af59-7fc0ca57832f service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:09.531821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5c76d-cd6a-4463-af59-7fc0ca57832f service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:09.590227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:09.591393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:09.591678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:09.591956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] PATH_INFO: `/v3/users/68bad7a1dd904461a9e0698751a739b7` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:09.593155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68bad7a1dd904461a9e0698751a739b7)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:09.595150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:09.644768 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68bad7a1dd904461a9e0698751a739b7'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:09.645419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68bad7a1dd904461a9e0698751a739b7'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:09.646016 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68bad7a1dd904461a9e0698751a739b7'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:09.653282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68bad7a1dd904461a9e0698751a739b7'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:09.660759 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9a90116c-e503-4118-95ee-2ae06fcb0635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68bad7a1dd904461a9e0698751a739b7'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:09.678997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1249/2510] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:09 2022] DELETE /identity/v3/users/68bad7a1dd904461a9e0698751a739b7 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:24:09.959822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:09.961381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:09.976558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc5bd17-a309-4a35-b1a2-1edb21241350 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.033845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.034729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.035014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.035320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] PATH_INFO: `/v3/projects/d7311482a4ad4af79ee0052ae9fbfec0` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.036477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7311482a4ad4af79ee0052ae9fbfec0)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.038416 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.085221 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7311482a4ad4af79ee0052ae9fbfec0', 'request_id': 'req-26caf18f-9447-4c33-a532-bac36c44109c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:10.092282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7311482a4ad4af79ee0052ae9fbfec0', 'request_id': 'req-26caf18f-9447-4c33-a532-bac36c44109c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:10.098372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7311482a4ad4af79ee0052ae9fbfec0', 'request_id': 'req-26caf18f-9447-4c33-a532-bac36c44109c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:10.104659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:10.109313 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] The token cache is being invalidate because project d7311482a4ad4af79ee0052ae9fbfec0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:10.110297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:10.111337 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-26caf18f-9447-4c33-a532-bac36c44109c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:10.112931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1262/2511] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:09 2022] DELETE /identity/v3/projects/d7311482a4ad4af79ee0052ae9fbfec0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:10.117917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21193a-997e-43e5-98d4-6fa202b07a4d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.118382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21193a-997e-43e5-98d4-6fa202b07a4d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.118479 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21193a-997e-43e5-98d4-6fa202b07a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.137090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-1e21193a-997e-43e5-98d4-6fa202b07a4d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:10.138275 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1e21193a-997e-43e5-98d4-6fa202b07a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:10.167033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:10.198020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1250/2512] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:10.206034 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7db32-49e6-441d-bb55-6444c19c33a2 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.206271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7db32-49e6-441d-bb55-6444c19c33a2 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.206555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf7db32-49e6-441d-bb55-6444c19c33a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.225607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-3cf7db32-49e6-441d-bb55-6444c19c33a2 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:10.226805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-3cf7db32-49e6-441d-bb55-6444c19c33a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:10.234823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1263/2513] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:10.238900 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21193a-997e-43e5-98d4-6fa202b07a4d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.248504 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21193a-997e-43e5-98d4-6fa202b07a4d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.253115 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16197a1d-8841-4e2d-bc0f-82f973b636ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.253765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16197a1d-8841-4e2d-bc0f-82f973b636ce admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.254047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16197a1d-8841-4e2d-bc0f-82f973b636ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.254324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16197a1d-8841-4e2d-bc0f-82f973b636ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.255127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16197a1d-8841-4e2d-bc0f-82f973b636ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:24:10.255388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16197a1d-8841-4e2d-bc0f-82f973b636ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.256998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16197a1d-8841-4e2d-bc0f-82f973b636ce admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.268842 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1251/2514] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:24:10.272874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7db32-49e6-441d-bb55-6444c19c33a2 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.282586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf7db32-49e6-441d-bb55-6444c19c33a2 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.287233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecec701-19fc-4c47-836a-2f3cad2402a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.287883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecec701-19fc-4c47-836a-2f3cad2402a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.288170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecec701-19fc-4c47-836a-2f3cad2402a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.288451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecec701-19fc-4c47-836a-2f3cad2402a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.289150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecec701-19fc-4c47-836a-2f3cad2402a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.291079 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecec701-19fc-4c47-836a-2f3cad2402a3 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.315119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1264/2515] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:10 2022] POST /identity/v3/projects => generated 377 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:10.319122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16197a1d-8841-4e2d-bc0f-82f973b636ce admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.329384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16197a1d-8841-4e2d-bc0f-82f973b636ce admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.383495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18806db4-e0ac-4cd6-9ea5-9df095194826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.384155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18806db4-e0ac-4cd6-9ea5-9df095194826 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.384441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18806db4-e0ac-4cd6-9ea5-9df095194826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.384721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18806db4-e0ac-4cd6-9ea5-9df095194826 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.385414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18806db4-e0ac-4cd6-9ea5-9df095194826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.387344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18806db4-e0ac-4cd6-9ea5-9df095194826 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.407755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1252/2516] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:10 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:10.411721 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecec701-19fc-4c47-836a-2f3cad2402a3 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.422033 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecec701-19fc-4c47-836a-2f3cad2402a3 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.426623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e15ec-ec64-47b5-97c5-e369a0930a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.427310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e15ec-ec64-47b5-97c5-e369a0930a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.427598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e15ec-ec64-47b5-97c5-e369a0930a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.427871 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e15ec-ec64-47b5-97c5-e369a0930a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.428584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e15ec-ec64-47b5-97c5-e369a0930a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.430186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e15ec-ec64-47b5-97c5-e369a0930a8f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.440815 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1265/2517] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:10.444531 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18806db4-e0ac-4cd6-9ea5-9df095194826 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.454808 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18806db4-e0ac-4cd6-9ea5-9df095194826 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.459509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f12d3b8-33a1-486d-b349-72b58392ab2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.460106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12d3b8-33a1-486d-b349-72b58392ab2d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.460391 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12d3b8-33a1-486d-b349-72b58392ab2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.460667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12d3b8-33a1-486d-b349-72b58392ab2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.461389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12d3b8-33a1-486d-b349-72b58392ab2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.462978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f12d3b8-33a1-486d-b349-72b58392ab2d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.473525 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1253/2518] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:10.477458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e15ec-ec64-47b5-97c5-e369a0930a8f admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.487385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e15ec-ec64-47b5-97c5-e369a0930a8f admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.492098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dc288-ca51-4be7-b4c0-cc00786f0cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.492967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dc288-ca51-4be7-b4c0-cc00786f0cec admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.493255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dc288-ca51-4be7-b4c0-cc00786f0cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.493535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83dc288-ca51-4be7-b4c0-cc00786f0cec admin admin] PATH_INFO: `/v3/projects/38415ed2c120468082ce0ab20b609ec6/users/f5ccf24b2c7344eb883ad78dc5fcde4f/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.508218 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83dc288-ca51-4be7-b4c0-cc00786f0cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=38415ed2c120468082ce0ab20b609ec6, user_id=f5ccf24b2c7344eb883ad78dc5fcde4f, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.510885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83dc288-ca51-4be7-b4c0-cc00786f0cec admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.524224 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1266/2519] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:10 2022] PUT /identity/v3/projects/38415ed2c120468082ce0ab20b609ec6/users/f5ccf24b2c7344eb883ad78dc5fcde4f/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:10.535175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99ea32-7dae-4a89-96fa-6655fabe5ea5 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.535476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99ea32-7dae-4a89-96fa-6655fabe5ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.535767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99ea32-7dae-4a89-96fa-6655fabe5ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.561497 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-fd99ea32-7dae-4a89-96fa-6655fabe5ea5 None None] MFA Rules not processed for user `f5ccf24b2c7344eb883ad78dc5fcde4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:10.563263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-fd99ea32-7dae-4a89-96fa-6655fabe5ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:10.627129 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1254/2520] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:10.631641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dc288-ca51-4be7-b4c0-cc00786f0cec admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.692542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83dc288-ca51-4be7-b4c0-cc00786f0cec admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.697130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63843c0b-47cf-4689-afe6-009970994a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.697779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63843c0b-47cf-4689-afe6-009970994a38 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.698066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63843c0b-47cf-4689-afe6-009970994a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.698346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63843c0b-47cf-4689-afe6-009970994a38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.699125 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63843c0b-47cf-4689-afe6-009970994a38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.701033 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63843c0b-47cf-4689-afe6-009970994a38 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.725823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1267/2521] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:10 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:10.729805 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99ea32-7dae-4a89-96fa-6655fabe5ea5 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.740354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99ea32-7dae-4a89-96fa-6655fabe5ea5 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.765798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:10.796152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9cbab-5354-4a5d-9408-be93e8cc8f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.796816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9cbab-5354-4a5d-9408-be93e8cc8f32 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.797127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9cbab-5354-4a5d-9408-be93e8cc8f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.797408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9cbab-5354-4a5d-9408-be93e8cc8f32 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.798094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9cbab-5354-4a5d-9408-be93e8cc8f32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.800183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9cbab-5354-4a5d-9408-be93e8cc8f32 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.819376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1255/2522] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:10 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:10.823325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63843c0b-47cf-4689-afe6-009970994a38 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.834351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63843c0b-47cf-4689-afe6-009970994a38 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.838885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73371709-da27-4450-9f0e-d49651fd0bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.839568 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73371709-da27-4450-9f0e-d49651fd0bae admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.839855 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73371709-da27-4450-9f0e-d49651fd0bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.840131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73371709-da27-4450-9f0e-d49651fd0bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.841095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73371709-da27-4450-9f0e-d49651fd0bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.842684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73371709-da27-4450-9f0e-d49651fd0bae admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.853979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1268/2523] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:10.858138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9cbab-5354-4a5d-9408-be93e8cc8f32 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.868527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9cbab-5354-4a5d-9408-be93e8cc8f32 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.873110 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729c479-f84f-4a2a-ae85-f981746087a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.873976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729c479-f84f-4a2a-ae85-f981746087a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.874266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729c479-f84f-4a2a-ae85-f981746087a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.874546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8729c479-f84f-4a2a-ae85-f981746087a2 admin admin] PATH_INFO: `/v3/projects/b721049501db42fdb948015a29b84daf/users/a842792257cd4f6c96e619637c66080f/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.890221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729c479-f84f-4a2a-ae85-f981746087a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b721049501db42fdb948015a29b84daf, user_id=a842792257cd4f6c96e619637c66080f, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.893772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8729c479-f84f-4a2a-ae85-f981746087a2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.906392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1256/2524] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:10 2022] PUT /identity/v3/projects/b721049501db42fdb948015a29b84daf/users/a842792257cd4f6c96e619637c66080f/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:10.910469 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73371709-da27-4450-9f0e-d49651fd0bae admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:10.920690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73371709-da27-4450-9f0e-d49651fd0bae admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:10.981729 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97024f98-e47a-4bf3-a31e-035d86023730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:10.982419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97024f98-e47a-4bf3-a31e-035d86023730 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:10.982615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97024f98-e47a-4bf3-a31e-035d86023730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:10.982894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97024f98-e47a-4bf3-a31e-035d86023730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:10.983690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97024f98-e47a-4bf3-a31e-035d86023730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:10.985787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97024f98-e47a-4bf3-a31e-035d86023730 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:10.995862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1269/2525] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:24:11.000090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729c479-f84f-4a2a-ae85-f981746087a2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:11.011334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8729c479-f84f-4a2a-ae85-f981746087a2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:11.015968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae65402-9d70-4839-abb1-462a429af382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:11.016868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae65402-9d70-4839-abb1-462a429af382 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:11.017153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae65402-9d70-4839-abb1-462a429af382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:11.017429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae65402-9d70-4839-abb1-462a429af382 admin admin] PATH_INFO: `/v3/projects/b721049501db42fdb948015a29b84daf/users/a842792257cd4f6c96e619637c66080f/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:11.019522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae65402-9d70-4839-abb1-462a429af382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b721049501db42fdb948015a29b84daf, user_id=a842792257cd4f6c96e619637c66080f, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:11.022142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae65402-9d70-4839-abb1-462a429af382 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:11.033901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1257/2526] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:10 2022] PUT /identity/v3/projects/b721049501db42fdb948015a29b84daf/users/a842792257cd4f6c96e619637c66080f/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:11.043827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2ccb8-9eed-4dd4-aa48-3788bc7c1637 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:11.044124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2ccb8-9eed-4dd4-aa48-3788bc7c1637 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:11.044475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2ccb8-9eed-4dd4-aa48-3788bc7c1637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:11.051858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:11.075068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ffa2ccb8-9eed-4dd4-aa48-3788bc7c1637 None None] MFA Rules not processed for user `a842792257cd4f6c96e619637c66080f`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:11.076328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ffa2ccb8-9eed-4dd4-aa48-3788bc7c1637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:11.144422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1270/2527] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:11 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:11.154812 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae65402-9d70-4839-abb1-462a429af382 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:11.156620 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8ae65402-9d70-4839-abb1-462a429af382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:11.213378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae65402-9d70-4839-abb1-462a429af382 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:11.217263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db5d4e-a27d-4669-a9b3-37674bdafe3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:11.217902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db5d4e-a27d-4669-a9b3-37674bdafe3f service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:11.218187 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db5d4e-a27d-4669-a9b3-37674bdafe3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:11.218467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db5d4e-a27d-4669-a9b3-37674bdafe3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:11.227824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db5d4e-a27d-4669-a9b3-37674bdafe3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:11.229258 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db5d4e-a27d-4669-a9b3-37674bdafe3f service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:11.239143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1258/2528] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:11.593299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2ccb8-9eed-4dd4-aa48-3788bc7c1637 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:11.655682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa2ccb8-9eed-4dd4-aa48-3788bc7c1637 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:11.660881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:11.661735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:11.662020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:11.662298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] PATH_INFO: `/v3/users/f5ccf24b2c7344eb883ad78dc5fcde4f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:11.663504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ccf24b2c7344eb883ad78dc5fcde4f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:11.665509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:11.708892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ccf24b2c7344eb883ad78dc5fcde4f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:11.709545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ccf24b2c7344eb883ad78dc5fcde4f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:11.710170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ccf24b2c7344eb883ad78dc5fcde4f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:11.717265 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ccf24b2c7344eb883ad78dc5fcde4f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:11.725422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ccf24b2c7344eb883ad78dc5fcde4f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:11.742757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1271/2529] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:11 2022] DELETE /identity/v3/users/f5ccf24b2c7344eb883ad78dc5fcde4f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:11.747216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe7058-297e-4ce0-9f9e-9a028b92f8df None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:11.747509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe7058-297e-4ce0-9f9e-9a028b92f8df None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:11.747786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe7058-297e-4ce0-9f9e-9a028b92f8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:11.764592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d6fe7058-297e-4ce0-9f9e-9a028b92f8df None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:11.766277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d6fe7058-297e-4ce0-9f9e-9a028b92f8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:11.822627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1259/2530] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:24:11.832646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:11.834080 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:11.845757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:11.896636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9803-fdcb-4df0-86f2-21c62dfb0dc0 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:11.900337 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb8c99-1aaa-42e5-878c-542f86a329b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:11.900993 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb8c99-1aaa-42e5-878c-542f86a329b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:11.901280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb8c99-1aaa-42e5-878c-542f86a329b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:11.901560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb8c99-1aaa-42e5-878c-542f86a329b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:11.911444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb8c99-1aaa-42e5-878c-542f86a329b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:11.912895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb8c99-1aaa-42e5-878c-542f86a329b8 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:11.923114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1272/2531] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:12.461585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe7058-297e-4ce0-9f9e-9a028b92f8df None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:12.463075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d6fe7058-297e-4ce0-9f9e-9a028b92f8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:12.474892 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe7058-297e-4ce0-9f9e-9a028b92f8df None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:12.480454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:12.481329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:12.481610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:12.481887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] PATH_INFO: `/v3/projects/38415ed2c120468082ce0ab20b609ec6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:12.483105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38415ed2c120468082ce0ab20b609ec6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:12.485512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:12.513122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:12.540219 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38415ed2c120468082ce0ab20b609ec6', 'request_id': 'req-9e631820-8312-428c-a98a-188451e70eb8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:12.550840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38415ed2c120468082ce0ab20b609ec6', 'request_id': 'req-9e631820-8312-428c-a98a-188451e70eb8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:12.558787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38415ed2c120468082ce0ab20b609ec6', 'request_id': 'req-9e631820-8312-428c-a98a-188451e70eb8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:12.569996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] The token cache is being invalidate because project 38415ed2c120468082ce0ab20b609ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:12.570287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:12.570938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:12.572492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1260/2532] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:12 2022] DELETE /identity/v3/projects/38415ed2c120468082ce0ab20b609ec6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:24:12.576549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb8c99-1aaa-42e5-878c-542f86a329b8 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:12.588192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb8c99-1aaa-42e5-878c-542f86a329b8 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:12.652544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:12.653425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:12.653717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:12.654027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] PATH_INFO: `/v3/users/a842792257cd4f6c96e619637c66080f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:12.655308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a842792257cd4f6c96e619637c66080f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:12.657785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:12.706164 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a842792257cd4f6c96e619637c66080f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:12.706894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a842792257cd4f6c96e619637c66080f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:12.707673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a842792257cd4f6c96e619637c66080f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:12.714596 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a842792257cd4f6c96e619637c66080f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:12.722937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-19d1793a-3232-42b4-b11e-d3829b0c3123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a842792257cd4f6c96e619637c66080f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:12.741961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1273/2533] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:12 2022] DELETE /identity/v3/users/a842792257cd4f6c96e619637c66080f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:13.048511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:13.051625 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:13.072582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e631820-8312-428c-a98a-188451e70eb8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:13.144715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:13.145737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.146201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.146516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] PATH_INFO: `/v3/projects/b721049501db42fdb948015a29b84daf` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.147918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b721049501db42fdb948015a29b84daf)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:13.150259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:13.201336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b721049501db42fdb948015a29b84daf', 'request_id': 'req-d55b1730-bdf0-4a9c-9370-38adb4e495c2'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:13.208715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b721049501db42fdb948015a29b84daf', 'request_id': 'req-d55b1730-bdf0-4a9c-9370-38adb4e495c2'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:13.215502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b721049501db42fdb948015a29b84daf', 'request_id': 'req-d55b1730-bdf0-4a9c-9370-38adb4e495c2'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:13.226555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] The token cache is being invalidate because project b721049501db42fdb948015a29b84daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:13.226847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:13.227523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-d55b1730-bdf0-4a9c-9370-38adb4e495c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:13.229066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1261/2534] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:13 2022] DELETE /identity/v3/projects/b721049501db42fdb948015a29b84daf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:13.235371 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e27253-bc3e-4495-9110-6261dfc9d84b None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.235732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e27253-bc3e-4495-9110-6261dfc9d84b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.236134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e27253-bc3e-4495-9110-6261dfc9d84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.254939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-61e27253-bc3e-4495-9110-6261dfc9d84b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:13.256182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-61e27253-bc3e-4495-9110-6261dfc9d84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:13.282827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:13.318732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1274/2535] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:13.328561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79515404-31c8-43a2-b8f6-71153b222dd8 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.328961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79515404-31c8-43a2-b8f6-71153b222dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.329310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79515404-31c8-43a2-b8f6-71153b222dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.348353 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-79515404-31c8-43a2-b8f6-71153b222dd8 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:13.349643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-79515404-31c8-43a2-b8f6-71153b222dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:13.358011 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1262/2536] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:13.362677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e27253-bc3e-4495-9110-6261dfc9d84b None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:13.373199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e27253-bc3e-4495-9110-6261dfc9d84b None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:13.378651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82166014-5b20-498e-b6b5-944527fd12a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:13.379400 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82166014-5b20-498e-b6b5-944527fd12a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.379785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82166014-5b20-498e-b6b5-944527fd12a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.380047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82166014-5b20-498e-b6b5-944527fd12a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.380844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82166014-5b20-498e-b6b5-944527fd12a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:24:13.381216 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82166014-5b20-498e-b6b5-944527fd12a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:13.383107 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82166014-5b20-498e-b6b5-944527fd12a8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:13.395633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1275/2537] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:24:13.400084 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79515404-31c8-43a2-b8f6-71153b222dd8 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:13.411064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79515404-31c8-43a2-b8f6-71153b222dd8 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:13.417234 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef38e6-c435-48b8-821f-feb57b978b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:13.417902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef38e6-c435-48b8-821f-feb57b978b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.418192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef38e6-c435-48b8-821f-feb57b978b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.418520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef38e6-c435-48b8-821f-feb57b978b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.419270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef38e6-c435-48b8-821f-feb57b978b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:13.421185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef38e6-c435-48b8-821f-feb57b978b9d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:13.460105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1263/2538] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:13 2022] POST /identity/v3/projects => generated 361 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:24:13.461222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82166014-5b20-498e-b6b5-944527fd12a8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:13.475851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82166014-5b20-498e-b6b5-944527fd12a8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:13.554271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bbb6c-f31d-4388-b7fc-18ac2bd3c960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:13.554954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bbb6c-f31d-4388-b7fc-18ac2bd3c960 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.555293 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bbb6c-f31d-4388-b7fc-18ac2bd3c960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.555574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944bbb6c-f31d-4388-b7fc-18ac2bd3c960 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.556284 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bbb6c-f31d-4388-b7fc-18ac2bd3c960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:13.558204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944bbb6c-f31d-4388-b7fc-18ac2bd3c960 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:13.585859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1276/2539] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:13 2022] POST /identity/v3/users => generated 281 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:13.590334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef38e6-c435-48b8-821f-feb57b978b9d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:13.602483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef38e6-c435-48b8-821f-feb57b978b9d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:13.607752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c845afe-cfb9-4ae8-9c68-76c2199d19b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:13.608423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c845afe-cfb9-4ae8-9c68-76c2199d19b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.608710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c845afe-cfb9-4ae8-9c68-76c2199d19b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.609131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c845afe-cfb9-4ae8-9c68-76c2199d19b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.610436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c845afe-cfb9-4ae8-9c68-76c2199d19b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:13.612090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c845afe-cfb9-4ae8-9c68-76c2199d19b0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:13.614501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:13.625186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1264/2540] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:13.628985 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bbb6c-f31d-4388-b7fc-18ac2bd3c960 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:13.641668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944bbb6c-f31d-4388-b7fc-18ac2bd3c960 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:13.646576 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67c205-479a-4a38-8b05-79a88a69a8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:13.647297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67c205-479a-4a38-8b05-79a88a69a8df admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.647612 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67c205-479a-4a38-8b05-79a88a69a8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.647899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67c205-479a-4a38-8b05-79a88a69a8df admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.648699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67c205-479a-4a38-8b05-79a88a69a8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:13.650321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67c205-479a-4a38-8b05-79a88a69a8df admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:13.662203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1277/2541] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:13.666444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c845afe-cfb9-4ae8-9c68-76c2199d19b0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:13.677304 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c845afe-cfb9-4ae8-9c68-76c2199d19b0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:13.681938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6bcc6-e60d-4a52-9af4-3b704374b714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:13.682823 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6bcc6-e60d-4a52-9af4-3b704374b714 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.683147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6bcc6-e60d-4a52-9af4-3b704374b714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.683423 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6bcc6-e60d-4a52-9af4-3b704374b714 admin admin] PATH_INFO: `/v3/projects/d5bacb56238d4fda887f46e8ab01f417/users/f56709dd568147ec84bac01289944250/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.698728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6bcc6-e60d-4a52-9af4-3b704374b714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5bacb56238d4fda887f46e8ab01f417, user_id=f56709dd568147ec84bac01289944250, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:13.701666 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6bcc6-e60d-4a52-9af4-3b704374b714 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:13.715419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1265/2542] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:13 2022] PUT /identity/v3/projects/d5bacb56238d4fda887f46e8ab01f417/users/f56709dd568147ec84bac01289944250/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:13.726579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43608e34-fe9c-448b-aac0-f11b30825ee4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.726874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43608e34-fe9c-448b-aac0-f11b30825ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.727232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43608e34-fe9c-448b-aac0-f11b30825ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.755623 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-43608e34-fe9c-448b-aac0-f11b30825ee4 None None] MFA Rules not processed for user `f56709dd568147ec84bac01289944250`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:13.756852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-43608e34-fe9c-448b-aac0-f11b30825ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:13.825213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1278/2543] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:13.856904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6bcc6-e60d-4a52-9af4-3b704374b714 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:13.858733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f7d6bcc6-e60d-4a52-9af4-3b704374b714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:13.920415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6bcc6-e60d-4a52-9af4-3b704374b714 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:13.925021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ef88f-7a76-4e96-9d1b-31e3ef118337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:13.925748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ef88f-7a76-4e96-9d1b-31e3ef118337 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:13.926106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ef88f-7a76-4e96-9d1b-31e3ef118337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:13.926454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ef88f-7a76-4e96-9d1b-31e3ef118337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:13.937050 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ef88f-7a76-4e96-9d1b-31e3ef118337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:13.939168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ef88f-7a76-4e96-9d1b-31e3ef118337 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:13.950262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1266/2544] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:13.965099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43608e34-fe9c-448b-aac0-f11b30825ee4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:13.995998 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:14.030290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43608e34-fe9c-448b-aac0-f11b30825ee4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:14.034899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:14.035820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:14.036114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:14.036398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] PATH_INFO: `/v3/users/f56709dd568147ec84bac01289944250` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:14.037608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f56709dd568147ec84bac01289944250)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:14.039585 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:14.086170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f56709dd568147ec84bac01289944250'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:14.086741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f56709dd568147ec84bac01289944250'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:14.087386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f56709dd568147ec84bac01289944250'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:14.094510 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f56709dd568147ec84bac01289944250'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:14.103199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f56709dd568147ec84bac01289944250'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:14.121084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1279/2545] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:13 2022] DELETE /identity/v3/users/f56709dd568147ec84bac01289944250 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:14.126229 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bcad6-d392-45a3-8800-f8ac927e64ae None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:14.126518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bcad6-d392-45a3-8800-f8ac927e64ae None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:14.126795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9bcad6-d392-45a3-8800-f8ac927e64ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:14.143912 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-3b9bcad6-d392-45a3-8800-f8ac927e64ae None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:14.145102 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3b9bcad6-d392-45a3-8800-f8ac927e64ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:14.201770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1267/2546] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:14.211273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:14.212703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:14.269723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e20c44-e01b-4134-8582-e752401bae35 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:14.273352 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ef2ac-5f66-4000-b42d-5634875cf1cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:14.273995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ef2ac-5f66-4000-b42d-5634875cf1cf service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:14.274283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ef2ac-5f66-4000-b42d-5634875cf1cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:14.274592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ef2ac-5f66-4000-b42d-5634875cf1cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:14.283570 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ef2ac-5f66-4000-b42d-5634875cf1cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:14.285013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ef2ac-5f66-4000-b42d-5634875cf1cf service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:14.294648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1280/2547] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:14.557722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bcad6-d392-45a3-8800-f8ac927e64ae None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:14.559208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3b9bcad6-d392-45a3-8800-f8ac927e64ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:14.571990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9bcad6-d392-45a3-8800-f8ac927e64ae None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:14.577138 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:14.577541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:14.577825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:14.578105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] PATH_INFO: `/v3/projects/d5bacb56238d4fda887f46e8ab01f417` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:14.580040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5bacb56238d4fda887f46e8ab01f417)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:14.581956 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:14.633010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5bacb56238d4fda887f46e8ab01f417', 'request_id': 'req-e4309403-67fc-470c-92f4-733b2f21c7c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:14.640601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5bacb56238d4fda887f46e8ab01f417', 'request_id': 'req-e4309403-67fc-470c-92f4-733b2f21c7c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:14.648322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5bacb56238d4fda887f46e8ab01f417', 'request_id': 'req-e4309403-67fc-470c-92f4-733b2f21c7c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:14.656712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:14.664421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] The token cache is being invalidate because project d5bacb56238d4fda887f46e8ab01f417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:14.665364 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:14.666608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e4309403-67fc-470c-92f4-733b2f21c7c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:14.669623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1268/2548] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:14 2022] DELETE /identity/v3/projects/d5bacb56238d4fda887f46e8ab01f417 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:14.676093 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5f360-ee9f-4a76-b7dd-a1b0b2e0c227 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:14.676699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5f360-ee9f-4a76-b7dd-a1b0b2e0c227 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:14.677176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5f360-ee9f-4a76-b7dd-a1b0b2e0c227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:14.697262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-64f5f360-ee9f-4a76-b7dd-a1b0b2e0c227 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:14.699057 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-64f5f360-ee9f-4a76-b7dd-a1b0b2e0c227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:14.746836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:14.765788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1281/2549] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:14.773226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e62ba-82ea-4332-af42-6ae50bec9d4e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:14.773596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e62ba-82ea-4332-af42-6ae50bec9d4e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:14.773948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608e62ba-82ea-4332-af42-6ae50bec9d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:14.794237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-608e62ba-82ea-4332-af42-6ae50bec9d4e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:14.795714 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-608e62ba-82ea-4332-af42-6ae50bec9d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:14.805633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1269/2550] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:24:14.809969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5f360-ee9f-4a76-b7dd-a1b0b2e0c227 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:14.821383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5f360-ee9f-4a76-b7dd-a1b0b2e0c227 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:14.826225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7abf00-3bdb-42ff-bd46-6db9b690dcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:14.826765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7abf00-3bdb-42ff-bd46-6db9b690dcac admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:14.827125 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7abf00-3bdb-42ff-bd46-6db9b690dcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:14.827553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7abf00-3bdb-42ff-bd46-6db9b690dcac admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:14.828393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7abf00-3bdb-42ff-bd46-6db9b690dcac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:24:14.828701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7abf00-3bdb-42ff-bd46-6db9b690dcac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:14.830383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7abf00-3bdb-42ff-bd46-6db9b690dcac admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:14.842717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1282/2551] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:24:14.847113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e62ba-82ea-4332-af42-6ae50bec9d4e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:14.856622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608e62ba-82ea-4332-af42-6ae50bec9d4e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:14.862039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab4fd2-e663-4127-b007-b9b2c47a0518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:14.862699 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab4fd2-e663-4127-b007-b9b2c47a0518 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:14.862980 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab4fd2-e663-4127-b007-b9b2c47a0518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:14.863288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab4fd2-e663-4127-b007-b9b2c47a0518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:14.863986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab4fd2-e663-4127-b007-b9b2c47a0518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:14.865954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab4fd2-e663-4127-b007-b9b2c47a0518 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:14.892072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1270/2552] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:14 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:24:14.896314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7abf00-3bdb-42ff-bd46-6db9b690dcac admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:14.906534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7abf00-3bdb-42ff-bd46-6db9b690dcac admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:14.965884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c40e17-a9ff-4e5b-a664-9b1eea9e8ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:14.966824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c40e17-a9ff-4e5b-a664-9b1eea9e8ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:14.967147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c40e17-a9ff-4e5b-a664-9b1eea9e8ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:14.967427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c40e17-a9ff-4e5b-a664-9b1eea9e8ed3 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:14.968117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c40e17-a9ff-4e5b-a664-9b1eea9e8ed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:14.970642 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c40e17-a9ff-4e5b-a664-9b1eea9e8ed3 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:14.993021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1283/2553] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:14 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:14.998590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab4fd2-e663-4127-b007-b9b2c47a0518 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:15.009761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab4fd2-e663-4127-b007-b9b2c47a0518 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:15.014836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4c463-f37d-4311-b345-b4ad67058f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:15.015567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4c463-f37d-4311-b345-b4ad67058f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.016416 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4c463-f37d-4311-b345-b4ad67058f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.016675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4c463-f37d-4311-b345-b4ad67058f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.017400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4c463-f37d-4311-b345-b4ad67058f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:15.019068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4c463-f37d-4311-b345-b4ad67058f44 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:15.029536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1271/2554] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:15.033673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c40e17-a9ff-4e5b-a664-9b1eea9e8ed3 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:15.043851 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c40e17-a9ff-4e5b-a664-9b1eea9e8ed3 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:15.048546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79909141-405c-4bd5-b75b-bba89e0672c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:15.049217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79909141-405c-4bd5-b75b-bba89e0672c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.049505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79909141-405c-4bd5-b75b-bba89e0672c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.049779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79909141-405c-4bd5-b75b-bba89e0672c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.050516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79909141-405c-4bd5-b75b-bba89e0672c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:15.052160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79909141-405c-4bd5-b75b-bba89e0672c0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:15.068324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4c463-f37d-4311-b345-b4ad67058f44 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:15.071447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1284/2555] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:15.082420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4c463-f37d-4311-b345-b4ad67058f44 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:15.087854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d0bd3-a70a-49eb-b298-ddd66e0a9807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:15.088751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d0bd3-a70a-49eb-b298-ddd66e0a9807 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.089034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d0bd3-a70a-49eb-b298-ddd66e0a9807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.089316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d0bd3-a70a-49eb-b298-ddd66e0a9807 admin admin] PATH_INFO: `/v3/projects/0fd2b9b3113d43928db3d86343144e78/users/7e6928107ce647919f90e9843d4cc9bb/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.104726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d0bd3-a70a-49eb-b298-ddd66e0a9807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd2b9b3113d43928db3d86343144e78, user_id=7e6928107ce647919f90e9843d4cc9bb, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:15.107447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d0bd3-a70a-49eb-b298-ddd66e0a9807 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:15.121094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1272/2556] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:15 2022] PUT /identity/v3/projects/0fd2b9b3113d43928db3d86343144e78/users/7e6928107ce647919f90e9843d4cc9bb/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:24:15.133282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ed49-0bec-450a-ba18-e3c15a25c000 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.133602 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ed49-0bec-450a-ba18-e3c15a25c000 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.133889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66ed49-0bec-450a-ba18-e3c15a25c000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.160738 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-af66ed49-0bec-450a-ba18-e3c15a25c000 None None] MFA Rules not processed for user `7e6928107ce647919f90e9843d4cc9bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:15.161932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-af66ed49-0bec-450a-ba18-e3c15a25c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:15.232806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1285/2557] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:15.237751 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d0bd3-a70a-49eb-b298-ddd66e0a9807 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:15.305510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d0bd3-a70a-49eb-b298-ddd66e0a9807 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:15.310298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e7808-8418-4055-ae32-a90563be11fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:15.310954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e7808-8418-4055-ae32-a90563be11fb admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.311277 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e7808-8418-4055-ae32-a90563be11fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.311558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e7808-8418-4055-ae32-a90563be11fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.312261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e7808-8418-4055-ae32-a90563be11fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:15.314218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e7808-8418-4055-ae32-a90563be11fb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:15.345188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1273/2558] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:15 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:15.346328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ed49-0bec-450a-ba18-e3c15a25c000 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:15.357493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66ed49-0bec-450a-ba18-e3c15a25c000 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:15.401999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:15.418658 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6405ba-ad88-41a7-9697-66d47cb1e4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:15.419296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6405ba-ad88-41a7-9697-66d47cb1e4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.419588 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6405ba-ad88-41a7-9697-66d47cb1e4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.419898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6405ba-ad88-41a7-9697-66d47cb1e4c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.420582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6405ba-ad88-41a7-9697-66d47cb1e4c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:15.422560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6405ba-ad88-41a7-9697-66d47cb1e4c5 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:15.441617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1286/2559] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:15 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:15.445596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e7808-8418-4055-ae32-a90563be11fb admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:15.456494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e7808-8418-4055-ae32-a90563be11fb admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:15.461366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965930d-be24-4b5b-ac13-449017f18ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:15.462024 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965930d-be24-4b5b-ac13-449017f18ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.462311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965930d-be24-4b5b-ac13-449017f18ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.462617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965930d-be24-4b5b-ac13-449017f18ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.463386 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965930d-be24-4b5b-ac13-449017f18ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:15.465007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965930d-be24-4b5b-ac13-449017f18ae6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:15.476381 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1274/2560] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:15.480473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6405ba-ad88-41a7-9697-66d47cb1e4c5 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:15.490449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6405ba-ad88-41a7-9697-66d47cb1e4c5 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:15.495140 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500a7c-f4ad-4f4c-b78f-9790bba2cf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:15.496014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500a7c-f4ad-4f4c-b78f-9790bba2cf5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.496303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500a7c-f4ad-4f4c-b78f-9790bba2cf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.496584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34500a7c-f4ad-4f4c-b78f-9790bba2cf5c admin admin] PATH_INFO: `/v3/projects/ecd3e81ff50448e385a82213476335e5/users/a20ceb5871564c4c995f5501990febcd/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.511174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34500a7c-f4ad-4f4c-b78f-9790bba2cf5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd3e81ff50448e385a82213476335e5, user_id=a20ceb5871564c4c995f5501990febcd, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:15.514503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34500a7c-f4ad-4f4c-b78f-9790bba2cf5c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:15.528621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1287/2561] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:15 2022] PUT /identity/v3/projects/ecd3e81ff50448e385a82213476335e5/users/a20ceb5871564c4c995f5501990febcd/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:15.533165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965930d-be24-4b5b-ac13-449017f18ae6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:15.542884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965930d-be24-4b5b-ac13-449017f18ae6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:15.597545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1662970-3a2d-4184-8179-25482e7707ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:15.598204 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1662970-3a2d-4184-8179-25482e7707ee admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.598487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1662970-3a2d-4184-8179-25482e7707ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.598761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1662970-3a2d-4184-8179-25482e7707ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.599540 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1662970-3a2d-4184-8179-25482e7707ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:15.601119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1662970-3a2d-4184-8179-25482e7707ee admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:15.610547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1275/2562] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:15.615100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500a7c-f4ad-4f4c-b78f-9790bba2cf5c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:15.625408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34500a7c-f4ad-4f4c-b78f-9790bba2cf5c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:15.630006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05403a58-5c1b-4114-8ec0-808e59ff1288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:15.630879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05403a58-5c1b-4114-8ec0-808e59ff1288 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.631206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05403a58-5c1b-4114-8ec0-808e59ff1288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.631493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05403a58-5c1b-4114-8ec0-808e59ff1288 admin admin] PATH_INFO: `/v3/projects/ecd3e81ff50448e385a82213476335e5/users/a20ceb5871564c4c995f5501990febcd/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.633505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05403a58-5c1b-4114-8ec0-808e59ff1288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecd3e81ff50448e385a82213476335e5, user_id=a20ceb5871564c4c995f5501990febcd, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:15.636199 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05403a58-5c1b-4114-8ec0-808e59ff1288 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:15.646818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1288/2563] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:15 2022] PUT /identity/v3/projects/ecd3e81ff50448e385a82213476335e5/users/a20ceb5871564c4c995f5501990febcd/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:15.657542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69169a9b-7d36-4ea3-97e3-96bbd833697f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.658036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69169a9b-7d36-4ea3-97e3-96bbd833697f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.658155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69169a9b-7d36-4ea3-97e3-96bbd833697f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.670707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:15.685046 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-69169a9b-7d36-4ea3-97e3-96bbd833697f None None] MFA Rules not processed for user `a20ceb5871564c4c995f5501990febcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:15.686274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-69169a9b-7d36-4ea3-97e3-96bbd833697f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:15.758588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1276/2564] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:24:15.769893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05403a58-5c1b-4114-8ec0-808e59ff1288 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:15.771748 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-05403a58-5c1b-4114-8ec0-808e59ff1288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:15.829208 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05403a58-5c1b-4114-8ec0-808e59ff1288 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:15.832895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c094eac-5e5f-48ce-bc5d-94de457a4dbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:15.833538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c094eac-5e5f-48ce-bc5d-94de457a4dbd service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:15.833825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c094eac-5e5f-48ce-bc5d-94de457a4dbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:15.834104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c094eac-5e5f-48ce-bc5d-94de457a4dbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:15.889897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c094eac-5e5f-48ce-bc5d-94de457a4dbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:15.891996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c094eac-5e5f-48ce-bc5d-94de457a4dbd service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:15.900711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1289/2565] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:18.395894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69169a9b-7d36-4ea3-97e3-96bbd833697f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:18.456658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69169a9b-7d36-4ea3-97e3-96bbd833697f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:18.461021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:18.461889 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:18.462167 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:18.462443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] PATH_INFO: `/v3/users/7e6928107ce647919f90e9843d4cc9bb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:18.463656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e6928107ce647919f90e9843d4cc9bb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:18.465566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:18.509014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e6928107ce647919f90e9843d4cc9bb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:18.509657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e6928107ce647919f90e9843d4cc9bb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:18.510260 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e6928107ce647919f90e9843d4cc9bb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:18.517068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e6928107ce647919f90e9843d4cc9bb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:18.524390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e6928107ce647919f90e9843d4cc9bb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:18.541837 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1277/2566] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:18 2022] DELETE /identity/v3/users/7e6928107ce647919f90e9843d4cc9bb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:18.546309 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8e32c-c30d-4f14-9414-7fad9a08ac74 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:18.546769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8e32c-c30d-4f14-9414-7fad9a08ac74 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:18.546887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8e32c-c30d-4f14-9414-7fad9a08ac74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:18.563383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-73c8e32c-c30d-4f14-9414-7fad9a08ac74 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:18.564595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-73c8e32c-c30d-4f14-9414-7fad9a08ac74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:18.610268 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:18.625144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1290/2567] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:24:18.634575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:18.635970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:18.649767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:18.695017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285b585e-d2f5-4127-a99e-878fd32c54e9 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:18.698632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e02c39-bd2e-4b2b-9285-a68b80dc45eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:18.699314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e02c39-bd2e-4b2b-9285-a68b80dc45eb service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:18.699601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e02c39-bd2e-4b2b-9285-a68b80dc45eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:18.699879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e02c39-bd2e-4b2b-9285-a68b80dc45eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:18.709322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e02c39-bd2e-4b2b-9285-a68b80dc45eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:18.710696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e02c39-bd2e-4b2b-9285-a68b80dc45eb service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:18.720238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1278/2568] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:18.953892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8e32c-c30d-4f14-9414-7fad9a08ac74 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:18.963727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8e32c-c30d-4f14-9414-7fad9a08ac74 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:18.967461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9352a-2ca1-4638-b12f-1816ebe59ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:18.968146 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9352a-2ca1-4638-b12f-1816ebe59ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:18.968435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9352a-2ca1-4638-b12f-1816ebe59ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:18.968714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c9352a-2ca1-4638-b12f-1816ebe59ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:18.971718 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-e7c9352a-2ca1-4638-b12f-1816ebe59ca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:18.984789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9352a-2ca1-4638-b12f-1816ebe59ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:18.986984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c9352a-2ca1-4638-b12f-1816ebe59ca5 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:18.995747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e02c39-bd2e-4b2b-9285-a68b80dc45eb service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:18.997248 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c5e02c39-bd2e-4b2b-9285-a68b80dc45eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:18.998135 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1291/2569] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:19.010911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e02c39-bd2e-4b2b-9285-a68b80dc45eb service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:19.017368 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:19.018295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:19.018640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:19.019708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] PATH_INFO: `/v3/projects/0fd2b9b3113d43928db3d86343144e78` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:19.021192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd2b9b3113d43928db3d86343144e78)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:19.023367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:19.082225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd2b9b3113d43928db3d86343144e78', 'request_id': 'req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:19.090012 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd2b9b3113d43928db3d86343144e78', 'request_id': 'req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:19.097403 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd2b9b3113d43928db3d86343144e78', 'request_id': 'req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:19.108920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] The token cache is being invalidate because project 0fd2b9b3113d43928db3d86343144e78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:19.109376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:19.110082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:19.111795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1279/2570] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:18 2022] DELETE /identity/v3/projects/0fd2b9b3113d43928db3d86343144e78 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:24:19.115926 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9352a-2ca1-4638-b12f-1816ebe59ca5 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:19.125711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c9352a-2ca1-4638-b12f-1816ebe59ca5 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:19.183283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:19.184389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:19.184685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:19.185058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] PATH_INFO: `/v3/users/a20ceb5871564c4c995f5501990febcd` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:19.186782 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20ceb5871564c4c995f5501990febcd)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:19.189735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:19.243590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20ceb5871564c4c995f5501990febcd'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:19.244178 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20ceb5871564c4c995f5501990febcd'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:19.244791 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20ceb5871564c4c995f5501990febcd'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:19.252381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20ceb5871564c4c995f5501990febcd'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:19.259321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-20c94333-4ab0-4a3d-adf1-cd4bfd2406ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20ceb5871564c4c995f5501990febcd'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:19.277385 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1292/2571] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:19 2022] DELETE /identity/v3/users/a20ceb5871564c4c995f5501990febcd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:19.955389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:19.956943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:19.969983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee68d2-48be-4cb1-8d55-ccf6c73e359c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:20.035374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:20.036213 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.036502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.036783 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] PATH_INFO: `/v3/projects/ecd3e81ff50448e385a82213476335e5` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.037984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecd3e81ff50448e385a82213476335e5)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:20.039967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:20.081466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:20.089918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecd3e81ff50448e385a82213476335e5', 'request_id': 'req-13609826-f0ba-4cb3-943d-c230903f85fe'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:20.098174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecd3e81ff50448e385a82213476335e5', 'request_id': 'req-13609826-f0ba-4cb3-943d-c230903f85fe'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:20.104433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecd3e81ff50448e385a82213476335e5', 'request_id': 'req-13609826-f0ba-4cb3-943d-c230903f85fe'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:20.115358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] The token cache is being invalidate because project ecd3e81ff50448e385a82213476335e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:20.115649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:20.116282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-13609826-f0ba-4cb3-943d-c230903f85fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:20.117820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1280/2572] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:19 2022] DELETE /identity/v3/projects/ecd3e81ff50448e385a82213476335e5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:20.122852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7805-561b-4f52-9d5c-601318e241ba None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.123391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7805-561b-4f52-9d5c-601318e241ba None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.123491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb7805-561b-4f52-9d5c-601318e241ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.141314 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-23bb7805-561b-4f52-9d5c-601318e241ba None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:20.142575 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-23bb7805-561b-4f52-9d5c-601318e241ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:20.195652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:20.205894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1293/2573] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:20.213623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea66e2-ad83-4014-9c1a-ae6b583ef774 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.213917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea66e2-ad83-4014-9c1a-ae6b583ef774 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.214202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea66e2-ad83-4014-9c1a-ae6b583ef774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.233513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-04ea66e2-ad83-4014-9c1a-ae6b583ef774 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:20.234725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-04ea66e2-ad83-4014-9c1a-ae6b583ef774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:20.243268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1281/2574] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:20.247699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb7805-561b-4f52-9d5c-601318e241ba None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:20.258382 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb7805-561b-4f52-9d5c-601318e241ba None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:20.263247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc2ab2-bada-470c-94c5-d88f56866ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:20.263983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc2ab2-bada-470c-94c5-d88f56866ded admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.264345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc2ab2-bada-470c-94c5-d88f56866ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.264750 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc2ab2-bada-470c-94c5-d88f56866ded admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.265664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc2ab2-bada-470c-94c5-d88f56866ded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:24:20.266048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc2ab2-bada-470c-94c5-d88f56866ded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:20.267790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc2ab2-bada-470c-94c5-d88f56866ded admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:20.285509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1294/2575] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:24:20.286667 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea66e2-ad83-4014-9c1a-ae6b583ef774 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:20.296851 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea66e2-ad83-4014-9c1a-ae6b583ef774 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:20.302071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaf515-2add-4dbe-a1bb-29338e8c4264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:20.302773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaf515-2add-4dbe-a1bb-29338e8c4264 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.303089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaf515-2add-4dbe-a1bb-29338e8c4264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.303379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaf515-2add-4dbe-a1bb-29338e8c4264 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.304090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceaf515-2add-4dbe-a1bb-29338e8c4264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:20.306051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceaf515-2add-4dbe-a1bb-29338e8c4264 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:20.337585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1282/2576] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:20 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:20.341773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc2ab2-bada-470c-94c5-d88f56866ded admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:20.352348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc2ab2-bada-470c-94c5-d88f56866ded admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:20.421834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd96a4b-6db2-4846-a162-1491ece58841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:20.422905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd96a4b-6db2-4846-a162-1491ece58841 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.423186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd96a4b-6db2-4846-a162-1491ece58841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.423634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd96a4b-6db2-4846-a162-1491ece58841 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.424356 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd96a4b-6db2-4846-a162-1491ece58841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:20.426376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd96a4b-6db2-4846-a162-1491ece58841 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:20.448778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1295/2577] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:20 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:20.452769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaf515-2add-4dbe-a1bb-29338e8c4264 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:20.464593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaf515-2add-4dbe-a1bb-29338e8c4264 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:20.469962 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc1321-4545-4a91-9537-a9d5161dae5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:20.470826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc1321-4545-4a91-9537-a9d5161dae5a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.471274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc1321-4545-4a91-9537-a9d5161dae5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.471555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc1321-4545-4a91-9537-a9d5161dae5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.472444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc1321-4545-4a91-9537-a9d5161dae5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:20.474061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc1321-4545-4a91-9537-a9d5161dae5a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:20.488379 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1283/2578] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:24:20.492292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd96a4b-6db2-4846-a162-1491ece58841 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:20.503692 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd96a4b-6db2-4846-a162-1491ece58841 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:20.508555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c9acf-8702-43d0-8b10-8e57f7f954a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:20.509223 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c9acf-8702-43d0-8b10-8e57f7f954a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.509514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c9acf-8702-43d0-8b10-8e57f7f954a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.509800 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c9acf-8702-43d0-8b10-8e57f7f954a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.510535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c9acf-8702-43d0-8b10-8e57f7f954a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:20.512220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c9acf-8702-43d0-8b10-8e57f7f954a4 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:20.526253 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1296/2579] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:24:20.527483 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc1321-4545-4a91-9537-a9d5161dae5a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:20.537715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc1321-4545-4a91-9537-a9d5161dae5a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:20.542311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e05d5-0496-4576-81db-4a65574162b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:20.543274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e05d5-0496-4576-81db-4a65574162b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.543511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e05d5-0496-4576-81db-4a65574162b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.543782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e05d5-0496-4576-81db-4a65574162b3 admin admin] PATH_INFO: `/v3/projects/a234142ee32e423ea63b5827e937c9cb/users/620b60d040bf4bbb8a6f3f358fc196c2/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.559144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e05d5-0496-4576-81db-4a65574162b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a234142ee32e423ea63b5827e937c9cb, user_id=620b60d040bf4bbb8a6f3f358fc196c2, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:20.561811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e05d5-0496-4576-81db-4a65574162b3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:20.576038 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1284/2580] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:20 2022] PUT /identity/v3/projects/a234142ee32e423ea63b5827e937c9cb/users/620b60d040bf4bbb8a6f3f358fc196c2/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:20.643930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471204a-496e-4c20-9dba-2c4d594f26d5 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.644239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471204a-496e-4c20-9dba-2c4d594f26d5 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.644654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471204a-496e-4c20-9dba-2c4d594f26d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.671989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-1471204a-496e-4c20-9dba-2c4d594f26d5 None None] MFA Rules not processed for user `620b60d040bf4bbb8a6f3f358fc196c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:20.673255 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-1471204a-496e-4c20-9dba-2c4d594f26d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:20.738490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1297/2581] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:20.743056 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e05d5-0496-4576-81db-4a65574162b3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:20.812029 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e05d5-0496-4576-81db-4a65574162b3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:20.817408 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135de1f-5c42-4f64-a0bc-b1e32b16ca62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:20.818091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135de1f-5c42-4f64-a0bc-b1e32b16ca62 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.818382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135de1f-5c42-4f64-a0bc-b1e32b16ca62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.818659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135de1f-5c42-4f64-a0bc-b1e32b16ca62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.819422 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135de1f-5c42-4f64-a0bc-b1e32b16ca62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:20.821356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135de1f-5c42-4f64-a0bc-b1e32b16ca62 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:20.851017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1285/2582] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:20 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:20.854887 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471204a-496e-4c20-9dba-2c4d594f26d5 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:20.866868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471204a-496e-4c20-9dba-2c4d594f26d5 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:20.932885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:20.944523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a99fe2-1030-4ff2-9583-c1eb11c0a358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:20.945399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a99fe2-1030-4ff2-9583-c1eb11c0a358 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.945783 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a99fe2-1030-4ff2-9583-c1eb11c0a358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.946159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a99fe2-1030-4ff2-9583-c1eb11c0a358 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.946950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a99fe2-1030-4ff2-9583-c1eb11c0a358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:20.949022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a99fe2-1030-4ff2-9583-c1eb11c0a358 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:20.975844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1298/2583] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:20 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:20.979761 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135de1f-5c42-4f64-a0bc-b1e32b16ca62 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:20.991473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135de1f-5c42-4f64-a0bc-b1e32b16ca62 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:20.997199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794ddec-5c3e-4f1d-9a5a-8e16b2b75998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:20.998005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794ddec-5c3e-4f1d-9a5a-8e16b2b75998 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:20.998674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794ddec-5c3e-4f1d-9a5a-8e16b2b75998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:20.998674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794ddec-5c3e-4f1d-9a5a-8e16b2b75998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:20.999604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794ddec-5c3e-4f1d-9a5a-8e16b2b75998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:21.001296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794ddec-5c3e-4f1d-9a5a-8e16b2b75998 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:21.012904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1286/2584] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:21.016739 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a99fe2-1030-4ff2-9583-c1eb11c0a358 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:21.027919 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a99fe2-1030-4ff2-9583-c1eb11c0a358 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:21.033406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b197a-394c-4c40-a605-67f01b917f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:21.034192 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b197a-394c-4c40-a605-67f01b917f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.034591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b197a-394c-4c40-a605-67f01b917f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.034977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b197a-394c-4c40-a605-67f01b917f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.035864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b197a-394c-4c40-a605-67f01b917f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:21.037574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b197a-394c-4c40-a605-67f01b917f8f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:21.049231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1299/2585] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:21.053499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794ddec-5c3e-4f1d-9a5a-8e16b2b75998 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:21.069964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794ddec-5c3e-4f1d-9a5a-8e16b2b75998 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:21.076437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dada9d4-d301-4a93-a751-f5a06fadd938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:21.079586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dada9d4-d301-4a93-a751-f5a06fadd938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.079586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dada9d4-d301-4a93-a751-f5a06fadd938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.079586 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dada9d4-d301-4a93-a751-f5a06fadd938 admin admin] PATH_INFO: `/v3/projects/f5e63e2a42b24df586e41755bc4610d6/users/0a62a8a7b0084b3e94f0703648f2462e/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.097144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dada9d4-d301-4a93-a751-f5a06fadd938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e63e2a42b24df586e41755bc4610d6, user_id=0a62a8a7b0084b3e94f0703648f2462e, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:21.099981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dada9d4-d301-4a93-a751-f5a06fadd938 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:21.115732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1287/2586] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:21 2022] PUT /identity/v3/projects/f5e63e2a42b24df586e41755bc4610d6/users/0a62a8a7b0084b3e94f0703648f2462e/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:24:21.125747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191988d-0138-44c4-91d4-03e5cc2706ae None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.126276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191988d-0138-44c4-91d4-03e5cc2706ae None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.126390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191988d-0138-44c4-91d4-03e5cc2706ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.155957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d191988d-0138-44c4-91d4-03e5cc2706ae None None] MFA Rules not processed for user `0a62a8a7b0084b3e94f0703648f2462e`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:21.157509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d191988d-0138-44c4-91d4-03e5cc2706ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:21.241964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1300/2587] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:21.246431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dada9d4-d301-4a93-a751-f5a06fadd938 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:21.263485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:21.318360 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dada9d4-d301-4a93-a751-f5a06fadd938 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:21.323433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4ab50-ef43-49f0-a6e9-d9e51402d21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:21.324131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ab50-ef43-49f0-a6e9-d9e51402d21c admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.324425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ab50-ef43-49f0-a6e9-d9e51402d21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.324711 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc4ab50-ef43-49f0-a6e9-d9e51402d21c admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.325778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4ab50-ef43-49f0-a6e9-d9e51402d21c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:21.327503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc4ab50-ef43-49f0-a6e9-d9e51402d21c admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:21.361558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1288/2588] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:21 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:24:21.362944 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191988d-0138-44c4-91d4-03e5cc2706ae None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:21.374876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191988d-0138-44c4-91d4-03e5cc2706ae None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:21.444482 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985cd64-e04c-4ea0-bb2a-70513d08bd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:21.445244 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985cd64-e04c-4ea0-bb2a-70513d08bd7a admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.445607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985cd64-e04c-4ea0-bb2a-70513d08bd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.445990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985cd64-e04c-4ea0-bb2a-70513d08bd7a admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.446786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985cd64-e04c-4ea0-bb2a-70513d08bd7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:21.448846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985cd64-e04c-4ea0-bb2a-70513d08bd7a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:21.472646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1301/2589] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:21 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:21.473857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4ab50-ef43-49f0-a6e9-d9e51402d21c admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:21.484468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc4ab50-ef43-49f0-a6e9-d9e51402d21c admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:21.489715 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922decc-71ff-4e91-895d-cb8f5fb1fe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:21.490441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922decc-71ff-4e91-895d-cb8f5fb1fe39 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.490797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922decc-71ff-4e91-895d-cb8f5fb1fe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.491214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922decc-71ff-4e91-895d-cb8f5fb1fe39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.492027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922decc-71ff-4e91-895d-cb8f5fb1fe39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:21.493680 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922decc-71ff-4e91-895d-cb8f5fb1fe39 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:21.508283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1289/2590] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:21.509551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985cd64-e04c-4ea0-bb2a-70513d08bd7a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:21.519931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985cd64-e04c-4ea0-bb2a-70513d08bd7a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:21.524741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98589f36-2305-4765-95e1-34d57553c2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:21.525634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98589f36-2305-4765-95e1-34d57553c2ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.525928 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98589f36-2305-4765-95e1-34d57553c2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.526214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98589f36-2305-4765-95e1-34d57553c2ac admin admin] PATH_INFO: `/v3/projects/b58ff65c50d6439ba0d45e27d308d129/users/851171cf24754f1fbb906ee7c72b9470/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.541744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98589f36-2305-4765-95e1-34d57553c2ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58ff65c50d6439ba0d45e27d308d129, user_id=851171cf24754f1fbb906ee7c72b9470, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:21.544802 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98589f36-2305-4765-95e1-34d57553c2ac admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:21.558348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1302/2591] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:21 2022] PUT /identity/v3/projects/b58ff65c50d6439ba0d45e27d308d129/users/851171cf24754f1fbb906ee7c72b9470/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:21.562443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922decc-71ff-4e91-895d-cb8f5fb1fe39 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:21.573396 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922decc-71ff-4e91-895d-cb8f5fb1fe39 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:21.636290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406089c5-e8bc-4e79-bd27-284c8cb2beb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:21.636974 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406089c5-e8bc-4e79-bd27-284c8cb2beb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.637283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406089c5-e8bc-4e79-bd27-284c8cb2beb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.637684 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406089c5-e8bc-4e79-bd27-284c8cb2beb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.638417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406089c5-e8bc-4e79-bd27-284c8cb2beb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:21.640051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406089c5-e8bc-4e79-bd27-284c8cb2beb5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:21.650523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1290/2592] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:24:21.654952 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98589f36-2305-4765-95e1-34d57553c2ac admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:21.666086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98589f36-2305-4765-95e1-34d57553c2ac admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:21.670812 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6ac47-a36d-4e7a-b43b-49d2a1990d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:21.671725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6ac47-a36d-4e7a-b43b-49d2a1990d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.672014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6ac47-a36d-4e7a-b43b-49d2a1990d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.672295 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6ac47-a36d-4e7a-b43b-49d2a1990d86 admin admin] PATH_INFO: `/v3/projects/b58ff65c50d6439ba0d45e27d308d129/users/851171cf24754f1fbb906ee7c72b9470/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.674303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6ac47-a36d-4e7a-b43b-49d2a1990d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b58ff65c50d6439ba0d45e27d308d129, user_id=851171cf24754f1fbb906ee7c72b9470, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:21.676996 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6ac47-a36d-4e7a-b43b-49d2a1990d86 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:21.687476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1303/2593] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:21 2022] PUT /identity/v3/projects/b58ff65c50d6439ba0d45e27d308d129/users/851171cf24754f1fbb906ee7c72b9470/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:21.699712 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215ac5a-6f1d-4875-8844-4b8879c6ba5a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.700003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215ac5a-6f1d-4875-8844-4b8879c6ba5a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.700283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8215ac5a-6f1d-4875-8844-4b8879c6ba5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.731621 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8215ac5a-6f1d-4875-8844-4b8879c6ba5a None None] MFA Rules not processed for user `851171cf24754f1fbb906ee7c72b9470`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:21.732936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8215ac5a-6f1d-4875-8844-4b8879c6ba5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:21.817350 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1291/2594] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:21.836177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6ac47-a36d-4e7a-b43b-49d2a1990d86 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:21.838099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8dd6ac47-a36d-4e7a-b43b-49d2a1990d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:21.853330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:21.911351 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6ac47-a36d-4e7a-b43b-49d2a1990d86 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:21.915298 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85b8f7-c7ab-4eab-8ece-c1b9329743d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:21.915956 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85b8f7-c7ab-4eab-8ece-c1b9329743d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:21.916249 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85b8f7-c7ab-4eab-8ece-c1b9329743d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:21.916529 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85b8f7-c7ab-4eab-8ece-c1b9329743d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:21.984908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85b8f7-c7ab-4eab-8ece-c1b9329743d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:21.986496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85b8f7-c7ab-4eab-8ece-c1b9329743d8 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:21.997579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1304/2595] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:22.472895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215ac5a-6f1d-4875-8844-4b8879c6ba5a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:22.474460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8215ac5a-6f1d-4875-8844-4b8879c6ba5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:22.487498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:22.553857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8215ac5a-6f1d-4875-8844-4b8879c6ba5a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:22.558318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:22.559271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:22.559555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:22.559834 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] PATH_INFO: `/v3/users/0187789947064682a8d9e685c2aadb9e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:22.573683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0187789947064682a8d9e685c2aadb9e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:22.575654 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:22.620914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0187789947064682a8d9e685c2aadb9e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:22.621522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0187789947064682a8d9e685c2aadb9e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:22.622100 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0187789947064682a8d9e685c2aadb9e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:22.630263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0187789947064682a8d9e685c2aadb9e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:22.637431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-59482c3e-55b9-4894-9c04-82bdaa948cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0187789947064682a8d9e685c2aadb9e'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:22.654672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1292/2596] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:22 2022] DELETE /identity/v3/users/0187789947064682a8d9e685c2aadb9e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:23.074475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85b8f7-c7ab-4eab-8ece-c1b9329743d8 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:23.086282 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85b8f7-c7ab-4eab-8ece-c1b9329743d8 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:23.144607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:23.145467 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.145753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.146032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] PATH_INFO: `/v3/projects/b8d16bd4e4864d3089a95df440ca5894` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.156648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d16bd4e4864d3089a95df440ca5894)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:23.159307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:23.204103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d16bd4e4864d3089a95df440ca5894', 'request_id': 'req-a6456540-f837-42e1-bcab-c6fb582a62ae'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:23.211398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d16bd4e4864d3089a95df440ca5894', 'request_id': 'req-a6456540-f837-42e1-bcab-c6fb582a62ae'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:23.218293 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d16bd4e4864d3089a95df440ca5894', 'request_id': 'req-a6456540-f837-42e1-bcab-c6fb582a62ae'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:23.219550 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:23.232232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] The token cache is being invalidate because project b8d16bd4e4864d3089a95df440ca5894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:23.232525 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:23.233162 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a6456540-f837-42e1-bcab-c6fb582a62ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:23.234735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1305/2597] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:23 2022] DELETE /identity/v3/projects/b8d16bd4e4864d3089a95df440ca5894 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:23.239731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce25b8-6950-4886-96aa-c80fdc3eb3bd None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.240027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce25b8-6950-4886-96aa-c80fdc3eb3bd None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.240307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce25b8-6950-4886-96aa-c80fdc3eb3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.259806 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-12ce25b8-6950-4886-96aa-c80fdc3eb3bd None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:23.261026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-12ce25b8-6950-4886-96aa-c80fdc3eb3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:23.318286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1293/2598] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:23.326879 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72461750-eceb-44ae-8c0a-7f90e1b5bd14 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.327210 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72461750-eceb-44ae-8c0a-7f90e1b5bd14 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.327497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72461750-eceb-44ae-8c0a-7f90e1b5bd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.345759 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-72461750-eceb-44ae-8c0a-7f90e1b5bd14 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:23.346974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-72461750-eceb-44ae-8c0a-7f90e1b5bd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:23.354920 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1306/2599] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:23.359145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce25b8-6950-4886-96aa-c80fdc3eb3bd None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:23.368786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce25b8-6950-4886-96aa-c80fdc3eb3bd None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:23.373269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5c3af-656a-4ec4-888d-d752ae868822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:23.373927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c3af-656a-4ec4-888d-d752ae868822 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.374211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c3af-656a-4ec4-888d-d752ae868822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.374492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5c3af-656a-4ec4-888d-d752ae868822 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.375269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5c3af-656a-4ec4-888d-d752ae868822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:24:23.375581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5c3af-656a-4ec4-888d-d752ae868822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:23.377725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5c3af-656a-4ec4-888d-d752ae868822 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:23.393202 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1294/2600] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:24:23.394362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72461750-eceb-44ae-8c0a-7f90e1b5bd14 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:23.404187 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72461750-eceb-44ae-8c0a-7f90e1b5bd14 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:23.409090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4e403-17be-4e37-9b4c-d3f89e136878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:23.409742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4e403-17be-4e37-9b4c-d3f89e136878 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.410029 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4e403-17be-4e37-9b4c-d3f89e136878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.410309 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4e403-17be-4e37-9b4c-d3f89e136878 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.411003 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4e403-17be-4e37-9b4c-d3f89e136878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:23.413019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4e403-17be-4e37-9b4c-d3f89e136878 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:23.448908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1307/2601] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:23 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:23.449977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5c3af-656a-4ec4-888d-d752ae868822 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:23.457038 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:23.461044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5c3af-656a-4ec4-888d-d752ae868822 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:23.520295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e633f3-585a-4db0-80fe-db6a09951db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:23.521022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e633f3-585a-4db0-80fe-db6a09951db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.521292 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e633f3-585a-4db0-80fe-db6a09951db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.521570 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e633f3-585a-4db0-80fe-db6a09951db1 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.522290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e633f3-585a-4db0-80fe-db6a09951db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:23.524530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e633f3-585a-4db0-80fe-db6a09951db1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:23.549626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1295/2602] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:23 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:23.551022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4e403-17be-4e37-9b4c-d3f89e136878 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:23.561763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4e403-17be-4e37-9b4c-d3f89e136878 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:23.567346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fe190-512c-4cc4-b2d1-b8b257e599d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:23.568060 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fe190-512c-4cc4-b2d1-b8b257e599d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.568348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fe190-512c-4cc4-b2d1-b8b257e599d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.568626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95fe190-512c-4cc4-b2d1-b8b257e599d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.569438 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fe190-512c-4cc4-b2d1-b8b257e599d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:23.569957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e633f3-585a-4db0-80fe-db6a09951db1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:23.571060 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95fe190-512c-4cc4-b2d1-b8b257e599d2 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:23.571635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-37e633f3-585a-4db0-80fe-db6a09951db1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:23.584271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1308/2603] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:23.585203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fe190-512c-4cc4-b2d1-b8b257e599d2 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:23.594757 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95fe190-512c-4cc4-b2d1-b8b257e599d2 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:23.599326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ed81f-22a4-439d-b97d-3ca6b52d9c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:23.600010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ed81f-22a4-439d-b97d-3ca6b52d9c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.600306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ed81f-22a4-439d-b97d-3ca6b52d9c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.600581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2ed81f-22a4-439d-b97d-3ca6b52d9c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.601285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ed81f-22a4-439d-b97d-3ca6b52d9c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:23.602847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2ed81f-22a4-439d-b97d-3ca6b52d9c9b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:23.613001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1309/2604] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:24:23.617042 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ed81f-22a4-439d-b97d-3ca6b52d9c9b admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:23.626980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2ed81f-22a4-439d-b97d-3ca6b52d9c9b admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:23.631524 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6412823-229c-4c80-82c0-36f85f32642a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:23.632408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6412823-229c-4c80-82c0-36f85f32642a admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.632673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6412823-229c-4c80-82c0-36f85f32642a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.633032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6412823-229c-4c80-82c0-36f85f32642a admin admin] PATH_INFO: `/v3/projects/4ef7c390e8014ca9906e82eef7fd387a/users/371e853f9ae3481dbd34a720803526c8/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.635436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e633f3-585a-4db0-80fe-db6a09951db1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:23.639091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d404d-e634-4803-b190-602de7c03c2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:23.639750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d404d-e634-4803-b190-602de7c03c2b service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.640033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d404d-e634-4803-b190-602de7c03c2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.640305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d404d-e634-4803-b190-602de7c03c2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.642657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-828d404d-e634-4803-b190-602de7c03c2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:23.648106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6412823-229c-4c80-82c0-36f85f32642a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef7c390e8014ca9906e82eef7fd387a, user_id=371e853f9ae3481dbd34a720803526c8, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:23.650751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6412823-229c-4c80-82c0-36f85f32642a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:23.664560 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1310/2605] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:23 2022] PUT /identity/v3/projects/4ef7c390e8014ca9906e82eef7fd387a/users/371e853f9ae3481dbd34a720803526c8/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:24:23.674708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf457885-40fe-40b5-89f8-eeb291104bb7 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.675472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf457885-40fe-40b5-89f8-eeb291104bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.675762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf457885-40fe-40b5-89f8-eeb291104bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.701635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-bf457885-40fe-40b5-89f8-eeb291104bb7 None None] MFA Rules not processed for user `371e853f9ae3481dbd34a720803526c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:23.701957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d404d-e634-4803-b190-602de7c03c2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:23.702869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bf457885-40fe-40b5-89f8-eeb291104bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:23.703492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828d404d-e634-4803-b190-602de7c03c2b service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:23.713731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1296/2606] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:24:23.782837 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1311/2607] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:23.787413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d404d-e634-4803-b190-602de7c03c2b service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:23.799388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d404d-e634-4803-b190-602de7c03c2b service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:23.847437 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:23.861708 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72232ae-4f10-445e-b6f6-59875ec1a7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:23.862661 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72232ae-4f10-445e-b6f6-59875ec1a7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.862954 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72232ae-4f10-445e-b6f6-59875ec1a7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.863279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72232ae-4f10-445e-b6f6-59875ec1a7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.864006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72232ae-4f10-445e-b6f6-59875ec1a7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:23.866414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72232ae-4f10-445e-b6f6-59875ec1a7e3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:23.892561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1297/2608] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:23 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:23.896682 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf457885-40fe-40b5-89f8-eeb291104bb7 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:23.907850 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf457885-40fe-40b5-89f8-eeb291104bb7 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:23.970753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b23901-3e28-4b5a-9cdf-4a18727fc3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:23.971625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b23901-3e28-4b5a-9cdf-4a18727fc3fc admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:23.972044 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b23901-3e28-4b5a-9cdf-4a18727fc3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:23.972493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b23901-3e28-4b5a-9cdf-4a18727fc3fc admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:23.973466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b23901-3e28-4b5a-9cdf-4a18727fc3fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:23.975760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b23901-3e28-4b5a-9cdf-4a18727fc3fc admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:23.984970 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:24.002901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1312/2609] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:23 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:24.004250 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72232ae-4f10-445e-b6f6-59875ec1a7e3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:24.016333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72232ae-4f10-445e-b6f6-59875ec1a7e3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:24.021604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731579-b189-4b78-b44e-7342b8ac8fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:24.022286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731579-b189-4b78-b44e-7342b8ac8fff admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:24.022569 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731579-b189-4b78-b44e-7342b8ac8fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:24.022854 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731579-b189-4b78-b44e-7342b8ac8fff admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:24.023629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731579-b189-4b78-b44e-7342b8ac8fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:24.025571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731579-b189-4b78-b44e-7342b8ac8fff admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:24.037106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1298/2610] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:24.041615 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b23901-3e28-4b5a-9cdf-4a18727fc3fc admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:24.055888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b23901-3e28-4b5a-9cdf-4a18727fc3fc admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:24.062604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41c944-ce9a-4651-bc28-7c1934c9f191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:24.063640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41c944-ce9a-4651-bc28-7c1934c9f191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:24.063994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41c944-ce9a-4651-bc28-7c1934c9f191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:24.064269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41c944-ce9a-4651-bc28-7c1934c9f191 admin admin] PATH_INFO: `/v3/projects/4f9a7f3a29c74c46ac39df8b525ae7db/users/4b7ce8bef3284dcea7edfcfedba0b51e/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:24.081859 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41c944-ce9a-4651-bc28-7c1934c9f191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9a7f3a29c74c46ac39df8b525ae7db, user_id=4b7ce8bef3284dcea7edfcfedba0b51e, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:24.084714 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41c944-ce9a-4651-bc28-7c1934c9f191 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:24.099737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1313/2611] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:24 2022] PUT /identity/v3/projects/4f9a7f3a29c74c46ac39df8b525ae7db/users/4b7ce8bef3284dcea7edfcfedba0b51e/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:24.103763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731579-b189-4b78-b44e-7342b8ac8fff admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:24.114172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731579-b189-4b78-b44e-7342b8ac8fff admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:24.170988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3e636-5f83-4ae4-b181-b88c92f713be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:24.171696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3e636-5f83-4ae4-b181-b88c92f713be admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:24.171983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3e636-5f83-4ae4-b181-b88c92f713be admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:24.172267 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3e636-5f83-4ae4-b181-b88c92f713be admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:24.172998 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3e636-5f83-4ae4-b181-b88c92f713be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:24.175271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3e636-5f83-4ae4-b181-b88c92f713be admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:24.185616 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1299/2612] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:24.189792 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41c944-ce9a-4651-bc28-7c1934c9f191 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:24.203936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41c944-ce9a-4651-bc28-7c1934c9f191 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:24.205572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ac665-93f3-4f77-8e6f-da3e6b43b3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:24.206509 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ac665-93f3-4f77-8e6f-da3e6b43b3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:24.206732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ac665-93f3-4f77-8e6f-da3e6b43b3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:24.207059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ac665-93f3-4f77-8e6f-da3e6b43b3a4 admin admin] PATH_INFO: `/v3/projects/4f9a7f3a29c74c46ac39df8b525ae7db/users/4b7ce8bef3284dcea7edfcfedba0b51e/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:24.209050 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ac665-93f3-4f77-8e6f-da3e6b43b3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f9a7f3a29c74c46ac39df8b525ae7db, user_id=4b7ce8bef3284dcea7edfcfedba0b51e, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:24.211747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ac665-93f3-4f77-8e6f-da3e6b43b3a4 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:24.231607 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1314/2613] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:24 2022] PUT /identity/v3/projects/4f9a7f3a29c74c46ac39df8b525ae7db/users/4b7ce8bef3284dcea7edfcfedba0b51e/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:24.233287 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e04d-a5a6-4ddf-b44c-05f2b9488989 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:24.233566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e04d-a5a6-4ddf-b44c-05f2b9488989 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:24.233847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e04d-a5a6-4ddf-b44c-05f2b9488989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:24.261894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cae2e04d-a5a6-4ddf-b44c-05f2b9488989 None None] MFA Rules not processed for user `4b7ce8bef3284dcea7edfcfedba0b51e`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:24.263163 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cae2e04d-a5a6-4ddf-b44c-05f2b9488989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:24.278539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ac665-93f3-4f77-8e6f-da3e6b43b3a4 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:24.334195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1300/2614] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:24.346019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2e04d-a5a6-4ddf-b44c-05f2b9488989 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:24.346291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ac665-93f3-4f77-8e6f-da3e6b43b3a4 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:24.347513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cae2e04d-a5a6-4ddf-b44c-05f2b9488989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:24.350001 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9401b98-1db3-47c6-80ea-14e513e73678 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:24.350727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9401b98-1db3-47c6-80ea-14e513e73678 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:24.351003 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9401b98-1db3-47c6-80ea-14e513e73678 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:24.351316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9401b98-1db3-47c6-80ea-14e513e73678 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:24.353100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f9401b98-1db3-47c6-80ea-14e513e73678 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:24.359631 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2e04d-a5a6-4ddf-b44c-05f2b9488989 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:24.363563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455ce9-628e-490d-9e3b-e6b4ef009f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:24.364256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455ce9-628e-490d-9e3b-e6b4ef009f9f service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:24.364549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455ce9-628e-490d-9e3b-e6b4ef009f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:24.364831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87455ce9-628e-490d-9e3b-e6b4ef009f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:24.408855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:24.418253 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9401b98-1db3-47c6-80ea-14e513e73678 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:24.419742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9401b98-1db3-47c6-80ea-14e513e73678 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:24.425678 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87455ce9-628e-490d-9e3b-e6b4ef009f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:24.427149 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87455ce9-628e-490d-9e3b-e6b4ef009f9f service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:24.429277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1315/2615] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:24:24.437543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1301/2616] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:24:30.926397 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9401b98-1db3-47c6-80ea-14e513e73678 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:30.934968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9401b98-1db3-47c6-80ea-14e513e73678 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:30.938640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed842f7-3948-4cd5-9fe5-4092152962a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:30.939304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed842f7-3948-4cd5-9fe5-4092152962a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:30.939595 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed842f7-3948-4cd5-9fe5-4092152962a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:30.939877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed842f7-3948-4cd5-9fe5-4092152962a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:30.941628 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-aed842f7-3948-4cd5-9fe5-4092152962a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:31.008144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:31.024860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed842f7-3948-4cd5-9fe5-4092152962a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:31.026250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed842f7-3948-4cd5-9fe5-4092152962a6 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:31.045049 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1316/2617] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:30 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:32.109083 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455ce9-628e-490d-9e3b-e6b4ef009f9f service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:32.118048 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87455ce9-628e-490d-9e3b-e6b4ef009f9f service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:32.121649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767db382-323a-4998-b4f7-bcf646e0db87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:32.122279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767db382-323a-4998-b4f7-bcf646e0db87 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:32.122563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767db382-323a-4998-b4f7-bcf646e0db87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:32.122839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767db382-323a-4998-b4f7-bcf646e0db87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:32.132416 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767db382-323a-4998-b4f7-bcf646e0db87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:32.133824 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767db382-323a-4998-b4f7-bcf646e0db87 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:32.143390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1302/2618] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:32 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:32.429666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed842f7-3948-4cd5-9fe5-4092152962a6 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:32.431226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-aed842f7-3948-4cd5-9fe5-4092152962a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:32.505669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed842f7-3948-4cd5-9fe5-4092152962a6 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:32.510239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:32.511143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:32.511428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:32.511705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] PATH_INFO: `/v3/users/620b60d040bf4bbb8a6f3f358fc196c2` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:32.512889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=620b60d040bf4bbb8a6f3f358fc196c2)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:32.514804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:32.568402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '620b60d040bf4bbb8a6f3f358fc196c2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:32.569670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '620b60d040bf4bbb8a6f3f358fc196c2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:32.570266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '620b60d040bf4bbb8a6f3f358fc196c2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:32.579658 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '620b60d040bf4bbb8a6f3f358fc196c2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:32.592513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '620b60d040bf4bbb8a6f3f358fc196c2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:32.615635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1317/2619] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:32 2022] DELETE /identity/v3/users/620b60d040bf4bbb8a6f3f358fc196c2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:32.620845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47419be1-7661-4c71-91aa-7ffd1d7be4f1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:32.621261 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47419be1-7661-4c71-91aa-7ffd1d7be4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:32.621550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47419be1-7661-4c71-91aa-7ffd1d7be4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:32.641853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-47419be1-7661-4c71-91aa-7ffd1d7be4f1 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:32.643109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-47419be1-7661-4c71-91aa-7ffd1d7be4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:32.703270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1303/2620] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:32.713248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:32.714672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:32.771020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b5c0e-c8cd-4250-8065-833e0830e357 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:32.774628 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5097b-0ec9-40ad-b1da-9ec8cf7535ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:32.775297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5097b-0ec9-40ad-b1da-9ec8cf7535ba service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:32.775583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5097b-0ec9-40ad-b1da-9ec8cf7535ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:32.775857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c5097b-0ec9-40ad-b1da-9ec8cf7535ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:32.785364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5097b-0ec9-40ad-b1da-9ec8cf7535ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:32.786774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c5097b-0ec9-40ad-b1da-9ec8cf7535ba service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:32.790640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:32.797212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1318/2621] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:33.061953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47419be1-7661-4c71-91aa-7ffd1d7be4f1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:33.063432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-47419be1-7661-4c71-91aa-7ffd1d7be4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:33.075748 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47419be1-7661-4c71-91aa-7ffd1d7be4f1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:33.080334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:33.081840 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:33.082120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:33.082392 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] PATH_INFO: `/v3/projects/a234142ee32e423ea63b5827e937c9cb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:33.083606 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a234142ee32e423ea63b5827e937c9cb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:33.085527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:33.130550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a234142ee32e423ea63b5827e937c9cb', 'request_id': 'req-39290d77-f4eb-4199-bf6e-98a14ec77013'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.137228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a234142ee32e423ea63b5827e937c9cb', 'request_id': 'req-39290d77-f4eb-4199-bf6e-98a14ec77013'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.143117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a234142ee32e423ea63b5827e937c9cb', 'request_id': 'req-39290d77-f4eb-4199-bf6e-98a14ec77013'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.153602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] The token cache is being invalidate because project a234142ee32e423ea63b5827e937c9cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:33.153890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.154512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.156030 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1304/2622] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:33 2022] DELETE /identity/v3/projects/a234142ee32e423ea63b5827e937c9cb => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:33.160045 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5097b-0ec9-40ad-b1da-9ec8cf7535ba service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:33.169419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c5097b-0ec9-40ad-b1da-9ec8cf7535ba service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:33.225400 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:33.226269 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:33.226549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:33.226823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] PATH_INFO: `/v3/users/0a62a8a7b0084b3e94f0703648f2462e` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:33.228031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a62a8a7b0084b3e94f0703648f2462e)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:33.229961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:33.273977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a62a8a7b0084b3e94f0703648f2462e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.274622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a62a8a7b0084b3e94f0703648f2462e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.275275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a62a8a7b0084b3e94f0703648f2462e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.282065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a62a8a7b0084b3e94f0703648f2462e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.288888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a62a8a7b0084b3e94f0703648f2462e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.306085 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1319/2623] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:33 2022] DELETE /identity/v3/users/0a62a8a7b0084b3e94f0703648f2462e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:33.577622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:33.579151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:33.594619 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39290d77-f4eb-4199-bf6e-98a14ec77013 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:33.643177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:33.653286 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:33.654157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:33.654726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:33.654953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] PATH_INFO: `/v3/projects/f5e63e2a42b24df586e41755bc4610d6` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:33.656144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e63e2a42b24df586e41755bc4610d6)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:33.658106 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:33.703901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e63e2a42b24df586e41755bc4610d6', 'request_id': 'req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.712134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e63e2a42b24df586e41755bc4610d6', 'request_id': 'req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.718925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e63e2a42b24df586e41755bc4610d6', 'request_id': 'req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.730224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] The token cache is being invalidate because project f5e63e2a42b24df586e41755bc4610d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:33.730521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.731210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.732875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1305/2624] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:33 2022] DELETE /identity/v3/projects/f5e63e2a42b24df586e41755bc4610d6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:33.737020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:33.738980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:33.752587 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893525b-e3e5-4595-bff8-ae58e54f7e57 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:33.811620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:33.812523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:33.812813 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:33.813093 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] PATH_INFO: `/v3/users/851171cf24754f1fbb906ee7c72b9470` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:33.814301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] RBAC: Authorizing `identity:delete_user(user_id=851171cf24754f1fbb906ee7c72b9470)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:33.816268 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:33.837264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:33.863744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '851171cf24754f1fbb906ee7c72b9470'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.864388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '851171cf24754f1fbb906ee7c72b9470'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.864985 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '851171cf24754f1fbb906ee7c72b9470'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.871728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '851171cf24754f1fbb906ee7c72b9470'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.878690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '851171cf24754f1fbb906ee7c72b9470'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:33.895331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1320/2625] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:33 2022] DELETE /identity/v3/users/851171cf24754f1fbb906ee7c72b9470 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:34.161112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:34.162626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:34.175765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f9bf0-cfab-4d3d-888b-a2d2a96d5736 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:34.234159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:34.235036 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:34.235351 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:34.235647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] PATH_INFO: `/v3/projects/b58ff65c50d6439ba0d45e27d308d129` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:34.236814 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b58ff65c50d6439ba0d45e27d308d129)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:34.238717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:34.282769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b58ff65c50d6439ba0d45e27d308d129', 'request_id': 'req-ceea7946-0dd9-4b82-a106-67e8c9570585'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:34.290203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b58ff65c50d6439ba0d45e27d308d129', 'request_id': 'req-ceea7946-0dd9-4b82-a106-67e8c9570585'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:34.296249 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b58ff65c50d6439ba0d45e27d308d129', 'request_id': 'req-ceea7946-0dd9-4b82-a106-67e8c9570585'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:34.306293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] The token cache is being invalidate because project b58ff65c50d6439ba0d45e27d308d129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:34.306589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:34.307236 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:34.308728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1306/2626] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:34 2022] DELETE /identity/v3/projects/b58ff65c50d6439ba0d45e27d308d129 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:43.891862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:DELETE=8,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:44.303352 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=78,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:45.795266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:45.796711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:45.853573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1527cf09-114b-464a-8c48-1e495399823a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:45.900890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b7916-f899-4089-8047-8db15dbe91f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:45.901546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b7916-f899-4089-8047-8db15dbe91f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:45.901834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b7916-f899-4089-8047-8db15dbe91f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:45.902141 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b7916-f899-4089-8047-8db15dbe91f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:45.903907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-915b7916-f899-4089-8047-8db15dbe91f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:45.972599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b7916-f899-4089-8047-8db15dbe91f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:45.973982 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b7916-f899-4089-8047-8db15dbe91f8 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:45.983601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1321/2627] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:48.785657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:48.787238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:48.804488 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea7946-0dd9-4b82-a106-67e8c9570585 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:48.809219 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:48.810131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:48.810575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:48.810858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] PATH_INFO: `/v3/users/8bbc454737084513baa37072cd160e73` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:48.812114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bbc454737084513baa37072cd160e73)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:48.814058 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:48.867359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bbc454737084513baa37072cd160e73'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:48.868010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bbc454737084513baa37072cd160e73'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:48.868608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bbc454737084513baa37072cd160e73'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:48.876342 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bbc454737084513baa37072cd160e73'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:48.883238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-29bbbd9b-87f1-4d04-9b47-b3526d847ab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bbc454737084513baa37072cd160e73'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:48.899797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1307/2628] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:48 2022] DELETE /identity/v3/users/8bbc454737084513baa37072cd160e73 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:49.074684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b7916-f899-4089-8047-8db15dbe91f8 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:49.085329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b7916-f899-4089-8047-8db15dbe91f8 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:49.145662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:49.146533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.146811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.147115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] PATH_INFO: `/v3/projects/6c0f74e2f5794c0189b647e29646eeb2` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.148295 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c0f74e2f5794c0189b647e29646eeb2)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:49.150200 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:49.158454 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:49.197328 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c0f74e2f5794c0189b647e29646eeb2', 'request_id': 'req-a4214cf6-0ced-45f7-b965-587add27f358'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:49.204910 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c0f74e2f5794c0189b647e29646eeb2', 'request_id': 'req-a4214cf6-0ced-45f7-b965-587add27f358'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:49.211386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c0f74e2f5794c0189b647e29646eeb2', 'request_id': 'req-a4214cf6-0ced-45f7-b965-587add27f358'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:49.221807 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] The token cache is being invalidate because project 6c0f74e2f5794c0189b647e29646eeb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:49.222096 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:49.222724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a4214cf6-0ced-45f7-b965-587add27f358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:49.224266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1322/2629] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:49 2022] DELETE /identity/v3/projects/6c0f74e2f5794c0189b647e29646eeb2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:49.229326 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176261-2a0f-4bba-b623-7cbdf16f4085 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.229618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176261-2a0f-4bba-b623-7cbdf16f4085 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.229898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5176261-2a0f-4bba-b623-7cbdf16f4085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.247968 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-c5176261-2a0f-4bba-b623-7cbdf16f4085 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:49.249165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-c5176261-2a0f-4bba-b623-7cbdf16f4085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:49.306926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1308/2630] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:49.314501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee7e0c-7923-4a95-abe0-0fdbe1415b15 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.314794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee7e0c-7923-4a95-abe0-0fdbe1415b15 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.315697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee7e0c-7923-4a95-abe0-0fdbe1415b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.334016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-64ee7e0c-7923-4a95-abe0-0fdbe1415b15 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:49.335252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-64ee7e0c-7923-4a95-abe0-0fdbe1415b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:49.343697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1323/2631] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:49.351965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5176261-2a0f-4bba-b623-7cbdf16f4085 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:49.361756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5176261-2a0f-4bba-b623-7cbdf16f4085 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:49.367785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36114040-0eb2-483f-b7a3-82be2d52e444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:49.368470 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36114040-0eb2-483f-b7a3-82be2d52e444 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.368757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36114040-0eb2-483f-b7a3-82be2d52e444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.369038 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36114040-0eb2-483f-b7a3-82be2d52e444 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.369796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36114040-0eb2-483f-b7a3-82be2d52e444 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:24:49.370109 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36114040-0eb2-483f-b7a3-82be2d52e444 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:49.371757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36114040-0eb2-483f-b7a3-82be2d52e444 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:49.383757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1309/2632] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:24:49.388055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee7e0c-7923-4a95-abe0-0fdbe1415b15 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:49.397170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee7e0c-7923-4a95-abe0-0fdbe1415b15 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:49.401719 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89ff8-68a0-4973-bf49-fc9c970679b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:49.402358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89ff8-68a0-4973-bf49-fc9c970679b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.402644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89ff8-68a0-4973-bf49-fc9c970679b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.402923 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb89ff8-68a0-4973-bf49-fc9c970679b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.403645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb89ff8-68a0-4973-bf49-fc9c970679b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:49.405559 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb89ff8-68a0-4973-bf49-fc9c970679b4 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:49.430684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1324/2633] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:49 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:49.434735 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36114040-0eb2-483f-b7a3-82be2d52e444 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:49.444777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36114040-0eb2-483f-b7a3-82be2d52e444 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:49.500003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca063d-64bf-480b-b831-8322fb955fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:49.500695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca063d-64bf-480b-b831-8322fb955fd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.501005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca063d-64bf-480b-b831-8322fb955fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.501288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca063d-64bf-480b-b831-8322fb955fd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.501984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca063d-64bf-480b-b831-8322fb955fd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:49.504002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca063d-64bf-480b-b831-8322fb955fd8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:49.510542 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:49.525467 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1310/2634] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:49 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:49.528876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89ff8-68a0-4973-bf49-fc9c970679b4 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:49.539541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb89ff8-68a0-4973-bf49-fc9c970679b4 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:49.544145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ded8c0-6726-4717-9e0b-070d6f81a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:49.544793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ded8c0-6726-4717-9e0b-070d6f81a6de admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.545077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ded8c0-6726-4717-9e0b-070d6f81a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.545355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ded8c0-6726-4717-9e0b-070d6f81a6de admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.546071 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ded8c0-6726-4717-9e0b-070d6f81a6de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:49.547708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ded8c0-6726-4717-9e0b-070d6f81a6de admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:49.558573 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1325/2635] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:49.562335 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca063d-64bf-480b-b831-8322fb955fd8 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:49.572253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca063d-64bf-480b-b831-8322fb955fd8 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:49.576817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2ee69-84b7-4f4c-9e2f-7c0cbe819c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:49.577471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2ee69-84b7-4f4c-9e2f-7c0cbe819c72 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.577762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2ee69-84b7-4f4c-9e2f-7c0cbe819c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.578044 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2ee69-84b7-4f4c-9e2f-7c0cbe819c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.578771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2ee69-84b7-4f4c-9e2f-7c0cbe819c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:49.580511 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b2ee69-84b7-4f4c-9e2f-7c0cbe819c72 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:49.590269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1311/2636] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:49.594561 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ded8c0-6726-4717-9e0b-070d6f81a6de admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:49.604973 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ded8c0-6726-4717-9e0b-070d6f81a6de admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:49.609628 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded6322-1df2-4e80-9a86-5fe1339697e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:49.610504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded6322-1df2-4e80-9a86-5fe1339697e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.610788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded6322-1df2-4e80-9a86-5fe1339697e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.611103 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ded6322-1df2-4e80-9a86-5fe1339697e5 admin admin] PATH_INFO: `/v3/projects/b405873047e644468ef700d501d84afa/users/90b5ab19ff2d45c0b4777fdb3df73c7d/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.626031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded6322-1df2-4e80-9a86-5fe1339697e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b405873047e644468ef700d501d84afa, user_id=90b5ab19ff2d45c0b4777fdb3df73c7d, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:49.628734 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ded6322-1df2-4e80-9a86-5fe1339697e5 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:49.641842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1326/2637] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:49 2022] PUT /identity/v3/projects/b405873047e644468ef700d501d84afa/users/90b5ab19ff2d45c0b4777fdb3df73c7d/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:49.652543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fefda-0e10-4928-8a9f-7b577bc67b2d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.652839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fefda-0e10-4928-8a9f-7b577bc67b2d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.653148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fefda-0e10-4928-8a9f-7b577bc67b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.680460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cf1fefda-0e10-4928-8a9f-7b577bc67b2d None None] MFA Rules not processed for user `90b5ab19ff2d45c0b4777fdb3df73c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:49.681691 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cf1fefda-0e10-4928-8a9f-7b577bc67b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:49.745970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1312/2638] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:49.756762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded6322-1df2-4e80-9a86-5fe1339697e5 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:49.758476 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2ded6322-1df2-4e80-9a86-5fe1339697e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:49.830237 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ded6322-1df2-4e80-9a86-5fe1339697e5 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:49.836334 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce0d08-0222-4d36-8ae5-dff9c59b3d38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:49.836980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce0d08-0222-4d36-8ae5-dff9c59b3d38 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:49.837268 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce0d08-0222-4d36-8ae5-dff9c59b3d38 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:49.837551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce0d08-0222-4d36-8ae5-dff9c59b3d38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:49.845566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce0d08-0222-4d36-8ae5-dff9c59b3d38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:49.847794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ce0d08-0222-4d36-8ae5-dff9c59b3d38 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:49.856594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1327/2639] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:53.576321 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fefda-0e10-4928-8a9f-7b577bc67b2d None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:53.638709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fefda-0e10-4928-8a9f-7b577bc67b2d None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:53.643830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:53.644698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:53.644985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:53.645264 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] PATH_INFO: `/v3/users/90b5ab19ff2d45c0b4777fdb3df73c7d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:53.646448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b5ab19ff2d45c0b4777fdb3df73c7d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:53.648411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:53.692856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b5ab19ff2d45c0b4777fdb3df73c7d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:53.693509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b5ab19ff2d45c0b4777fdb3df73c7d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:53.694107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b5ab19ff2d45c0b4777fdb3df73c7d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:53.701166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b5ab19ff2d45c0b4777fdb3df73c7d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:53.708433 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b5ab19ff2d45c0b4777fdb3df73c7d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:53.713475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:53.727779 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1313/2640] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:53 2022] DELETE /identity/v3/users/90b5ab19ff2d45c0b4777fdb3df73c7d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:53.732496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3783ca-a3cc-4cf8-86dc-15aacbd25d3e None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:53.732988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3783ca-a3cc-4cf8-86dc-15aacbd25d3e None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:53.733100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3783ca-a3cc-4cf8-86dc-15aacbd25d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:53.749892 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-9f3783ca-a3cc-4cf8-86dc-15aacbd25d3e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:53.751159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9f3783ca-a3cc-4cf8-86dc-15aacbd25d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:53.763315 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:53.812362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1328/2641] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:53.821897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:53.823336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:53.880528 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3ff78-7622-400b-bb59-da6e1433ed72 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:53.884168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b175aa-38bc-416d-b19b-4e43fb1c505d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:53.884817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b175aa-38bc-416d-b19b-4e43fb1c505d service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:53.885107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b175aa-38bc-416d-b19b-4e43fb1c505d service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:53.885393 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b175aa-38bc-416d-b19b-4e43fb1c505d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:53.894838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b175aa-38bc-416d-b19b-4e43fb1c505d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:53.896306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b175aa-38bc-416d-b19b-4e43fb1c505d service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:53.905724 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1314/2642] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:54.167386 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3783ca-a3cc-4cf8-86dc-15aacbd25d3e None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:54.168857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9f3783ca-a3cc-4cf8-86dc-15aacbd25d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:54.183690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3783ca-a3cc-4cf8-86dc-15aacbd25d3e None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:54.188277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:54.189139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.189424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.189699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] PATH_INFO: `/v3/projects/b405873047e644468ef700d501d84afa` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.191534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b405873047e644468ef700d501d84afa)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:54.193454 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:54.239662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b405873047e644468ef700d501d84afa', 'request_id': 'req-d064b509-0922-4580-b7e6-a209e5e86dd9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:54.246676 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b405873047e644468ef700d501d84afa', 'request_id': 'req-d064b509-0922-4580-b7e6-a209e5e86dd9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:54.253617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b405873047e644468ef700d501d84afa', 'request_id': 'req-d064b509-0922-4580-b7e6-a209e5e86dd9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:54.264012 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] The token cache is being invalidate because project b405873047e644468ef700d501d84afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:54.264301 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:54.264931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d064b509-0922-4580-b7e6-a209e5e86dd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:54.266483 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1329/2643] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:54 2022] DELETE /identity/v3/projects/b405873047e644468ef700d501d84afa => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:54.272042 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd618f1f-4d67-4636-9dc5-a6a6c1045b2a None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.272332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd618f1f-4d67-4636-9dc5-a6a6c1045b2a None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.272613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd618f1f-4d67-4636-9dc5-a6a6c1045b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.289977 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-cd618f1f-4d67-4636-9dc5-a6a6c1045b2a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:54.291191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cd618f1f-4d67-4636-9dc5-a6a6c1045b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:54.350183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1315/2644] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:54.357712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91c45-824e-4439-a65c-01a1d001b7ae None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.357983 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91c45-824e-4439-a65c-01a1d001b7ae None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.358306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d91c45-824e-4439-a65c-01a1d001b7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.377358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b9d91c45-824e-4439-a65c-01a1d001b7ae None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:54.379263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b9d91c45-824e-4439-a65c-01a1d001b7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:54.387188 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1330/2645] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:54.391458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd618f1f-4d67-4636-9dc5-a6a6c1045b2a None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:54.401246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd618f1f-4d67-4636-9dc5-a6a6c1045b2a None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:54.405859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d95826-62ab-48c5-b9ec-a563aea75ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:54.406513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d95826-62ab-48c5-b9ec-a563aea75ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.406800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d95826-62ab-48c5-b9ec-a563aea75ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.407111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d95826-62ab-48c5-b9ec-a563aea75ec3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.407844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d95826-62ab-48c5-b9ec-a563aea75ec3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:24:54.408157 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d95826-62ab-48c5-b9ec-a563aea75ec3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:54.409753 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d95826-62ab-48c5-b9ec-a563aea75ec3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:54.421210 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1316/2646] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:24:54.425761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d91c45-824e-4439-a65c-01a1d001b7ae None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:54.434965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d91c45-824e-4439-a65c-01a1d001b7ae None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:54.439753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db763d-8e5b-4eab-8c74-3af3c4b8aef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:54.441047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db763d-8e5b-4eab-8c74-3af3c4b8aef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.441330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db763d-8e5b-4eab-8c74-3af3c4b8aef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.441606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db763d-8e5b-4eab-8c74-3af3c4b8aef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.442289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db763d-8e5b-4eab-8c74-3af3c4b8aef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:54.444300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db763d-8e5b-4eab-8c74-3af3c4b8aef3 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:54.469632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1331/2647] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:54 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:54.473583 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d95826-62ab-48c5-b9ec-a563aea75ec3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:54.483583 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d95826-62ab-48c5-b9ec-a563aea75ec3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:54.538004 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89627c-430d-46ad-b0bb-70630e0e9f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:54.538664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89627c-430d-46ad-b0bb-70630e0e9f70 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.538951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89627c-430d-46ad-b0bb-70630e0e9f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.539313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89627c-430d-46ad-b0bb-70630e0e9f70 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.540005 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89627c-430d-46ad-b0bb-70630e0e9f70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:54.541918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89627c-430d-46ad-b0bb-70630e0e9f70 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:54.563636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1317/2648] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:54 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:54.568261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db763d-8e5b-4eab-8c74-3af3c4b8aef3 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:54.578513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db763d-8e5b-4eab-8c74-3af3c4b8aef3 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:54.583163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa1c5c-d17a-40ca-a39e-f1be609a3809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:54.583819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa1c5c-d17a-40ca-a39e-f1be609a3809 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.584105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa1c5c-d17a-40ca-a39e-f1be609a3809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.584383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa1c5c-d17a-40ca-a39e-f1be609a3809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.585104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa1c5c-d17a-40ca-a39e-f1be609a3809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:54.586704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa1c5c-d17a-40ca-a39e-f1be609a3809 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:54.597231 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1332/2649] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:54.601031 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89627c-430d-46ad-b0bb-70630e0e9f70 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:54.610601 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89627c-430d-46ad-b0bb-70630e0e9f70 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:54.615729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e9d2d-86a0-418a-83c3-9ac5b538ae68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:54.616384 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e9d2d-86a0-418a-83c3-9ac5b538ae68 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.616676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e9d2d-86a0-418a-83c3-9ac5b538ae68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.616959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e9d2d-86a0-418a-83c3-9ac5b538ae68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.617681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e9d2d-86a0-418a-83c3-9ac5b538ae68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:54.619317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e9d2d-86a0-418a-83c3-9ac5b538ae68 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:54.620548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:54.631952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1318/2650] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:54.635705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa1c5c-d17a-40ca-a39e-f1be609a3809 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:54.645668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa1c5c-d17a-40ca-a39e-f1be609a3809 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:54.650533 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2dc8a-e894-4f44-b512-c1d8af4c71da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:54.651446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2dc8a-e894-4f44-b512-c1d8af4c71da admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.651732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2dc8a-e894-4f44-b512-c1d8af4c71da admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.652010 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2dc8a-e894-4f44-b512-c1d8af4c71da admin admin] PATH_INFO: `/v3/projects/46395296793949c29ea77ffe0a8428db/users/b59346323b974858a92ed39bbe066846/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.666794 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2dc8a-e894-4f44-b512-c1d8af4c71da admin admin] RBAC: Authorizing `identity:create_grant(project_id=46395296793949c29ea77ffe0a8428db, user_id=b59346323b974858a92ed39bbe066846, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:54.669551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2dc8a-e894-4f44-b512-c1d8af4c71da admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:54.683104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1333/2651] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:54 2022] PUT /identity/v3/projects/46395296793949c29ea77ffe0a8428db/users/b59346323b974858a92ed39bbe066846/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:24:54.693227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae3ec2-df60-4a9f-b499-ea13c448f55e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.693522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae3ec2-df60-4a9f-b499-ea13c448f55e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.693830 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae3ec2-df60-4a9f-b499-ea13c448f55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.719247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-caae3ec2-df60-4a9f-b499-ea13c448f55e None None] MFA Rules not processed for user `b59346323b974858a92ed39bbe066846`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:54.720448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-caae3ec2-df60-4a9f-b499-ea13c448f55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:54.783446 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1319/2652] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:54.788273 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2dc8a-e894-4f44-b512-c1d8af4c71da admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:54.829343 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:54.852021 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2dc8a-e894-4f44-b512-c1d8af4c71da admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:54.856740 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0afe1-068e-49ad-8976-b79ca603f02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:54.857403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0afe1-068e-49ad-8976-b79ca603f02c admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.857696 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0afe1-068e-49ad-8976-b79ca603f02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.857979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0afe1-068e-49ad-8976-b79ca603f02c admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.858672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0afe1-068e-49ad-8976-b79ca603f02c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:54.860899 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0afe1-068e-49ad-8976-b79ca603f02c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:54.885213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1334/2653] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:54 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:54.889259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae3ec2-df60-4a9f-b499-ea13c448f55e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:54.900002 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae3ec2-df60-4a9f-b499-ea13c448f55e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:54.959906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4beb2c4-86ab-44d8-b80f-7a381f574025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:54.960592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4beb2c4-86ab-44d8-b80f-7a381f574025 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:54.960878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4beb2c4-86ab-44d8-b80f-7a381f574025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:54.961155 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4beb2c4-86ab-44d8-b80f-7a381f574025 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:54.961849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4beb2c4-86ab-44d8-b80f-7a381f574025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:54.963822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4beb2c4-86ab-44d8-b80f-7a381f574025 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:54.984209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1320/2654] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:54 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:54.988861 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0afe1-068e-49ad-8976-b79ca603f02c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:54.999229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0afe1-068e-49ad-8976-b79ca603f02c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:55.003950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6db68-5e66-45b5-812e-a9d8a0ad2125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:55.004617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6db68-5e66-45b5-812e-a9d8a0ad2125 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:55.004909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6db68-5e66-45b5-812e-a9d8a0ad2125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:55.005194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6db68-5e66-45b5-812e-a9d8a0ad2125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:55.005922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6db68-5e66-45b5-812e-a9d8a0ad2125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:55.007574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6db68-5e66-45b5-812e-a9d8a0ad2125 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:55.017980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1335/2655] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:55.022327 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4beb2c4-86ab-44d8-b80f-7a381f574025 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:55.031984 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4beb2c4-86ab-44d8-b80f-7a381f574025 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:55.037059 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b5061-d4cc-4b6a-a328-f6e839329216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:55.037924 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b5061-d4cc-4b6a-a328-f6e839329216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:55.038222 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b5061-d4cc-4b6a-a328-f6e839329216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:55.038487 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b5061-d4cc-4b6a-a328-f6e839329216 admin admin] PATH_INFO: `/v3/projects/31dcfa6930ab4424a404cb18749d5521/users/60fbd86fe5b34828a7248e350d98fac8/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:55.052856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b5061-d4cc-4b6a-a328-f6e839329216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31dcfa6930ab4424a404cb18749d5521, user_id=60fbd86fe5b34828a7248e350d98fac8, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:55.056052 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b5061-d4cc-4b6a-a328-f6e839329216 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:55.070341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1321/2656] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:55 2022] PUT /identity/v3/projects/31dcfa6930ab4424a404cb18749d5521/users/60fbd86fe5b34828a7248e350d98fac8/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:24:55.074695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6db68-5e66-45b5-812e-a9d8a0ad2125 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:55.086918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6db68-5e66-45b5-812e-a9d8a0ad2125 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:55.141962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da72cc-9a51-4c41-accd-6a0e70e6439a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:55.142634 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da72cc-9a51-4c41-accd-6a0e70e6439a admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:55.142927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da72cc-9a51-4c41-accd-6a0e70e6439a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:55.143731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da72cc-9a51-4c41-accd-6a0e70e6439a admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:55.144459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da72cc-9a51-4c41-accd-6a0e70e6439a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:55.146065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63da72cc-9a51-4c41-accd-6a0e70e6439a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:55.155693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1336/2657] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:55.160552 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b5061-d4cc-4b6a-a328-f6e839329216 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:55.170660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b5061-d4cc-4b6a-a328-f6e839329216 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:55.175274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716c7ad-38ac-458a-89ed-a7d54e92431e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:55.176154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716c7ad-38ac-458a-89ed-a7d54e92431e admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:55.176442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716c7ad-38ac-458a-89ed-a7d54e92431e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:55.176720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3716c7ad-38ac-458a-89ed-a7d54e92431e admin admin] PATH_INFO: `/v3/projects/31dcfa6930ab4424a404cb18749d5521/users/60fbd86fe5b34828a7248e350d98fac8/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:55.178676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716c7ad-38ac-458a-89ed-a7d54e92431e admin admin] RBAC: Authorizing `identity:create_grant(project_id=31dcfa6930ab4424a404cb18749d5521, user_id=60fbd86fe5b34828a7248e350d98fac8, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:55.181318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3716c7ad-38ac-458a-89ed-a7d54e92431e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:55.191957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1322/2658] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:55 2022] PUT /identity/v3/projects/31dcfa6930ab4424a404cb18749d5521/users/60fbd86fe5b34828a7248e350d98fac8/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:55.203557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da4721-5b72-4da9-b009-ce40fda0dfb3 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:55.203862 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da4721-5b72-4da9-b009-ce40fda0dfb3 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:55.204152 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da4721-5b72-4da9-b009-ce40fda0dfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:55.231562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-a6da4721-5b72-4da9-b009-ce40fda0dfb3 None None] MFA Rules not processed for user `60fbd86fe5b34828a7248e350d98fac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:55.232776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a6da4721-5b72-4da9-b009-ce40fda0dfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:55.299338 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1337/2659] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:55 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:24:55.310313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716c7ad-38ac-458a-89ed-a7d54e92431e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:55.312064 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-3716c7ad-38ac-458a-89ed-a7d54e92431e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:55.354469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:55.370491 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3716c7ad-38ac-458a-89ed-a7d54e92431e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:55.374471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ad72c-dc1b-4d20-a57c-2613ab5c97de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:55.375156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ad72c-dc1b-4d20-a57c-2613ab5c97de service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:55.375450 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ad72c-dc1b-4d20-a57c-2613ab5c97de service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:55.375760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ad72c-dc1b-4d20-a57c-2613ab5c97de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:55.384737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ad72c-dc1b-4d20-a57c-2613ab5c97de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:55.386176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7ad72c-dc1b-4d20-a57c-2613ab5c97de service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:55.396480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1323/2660] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:55 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:55.784305 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da4721-5b72-4da9-b009-ce40fda0dfb3 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:55.818125 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:55.845872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da4721-5b72-4da9-b009-ce40fda0dfb3 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:55.850523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:55.851413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:55.851694 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:55.851967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] PATH_INFO: `/v3/users/b59346323b974858a92ed39bbe066846` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:55.853139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b59346323b974858a92ed39bbe066846)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:55.855066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:55.900295 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b59346323b974858a92ed39bbe066846'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:55.900876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b59346323b974858a92ed39bbe066846'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:55.901482 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b59346323b974858a92ed39bbe066846'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:55.908312 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b59346323b974858a92ed39bbe066846'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:55.915370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b59346323b974858a92ed39bbe066846'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:55.932760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1338/2661] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:55 2022] DELETE /identity/v3/users/b59346323b974858a92ed39bbe066846 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:55.937430 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d64c6-530c-4a27-82c9-3a35f5f4202f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:55.937881 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d64c6-530c-4a27-82c9-3a35f5f4202f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:55.938001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d64c6-530c-4a27-82c9-3a35f5f4202f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:55.956459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-222d64c6-530c-4a27-82c9-3a35f5f4202f None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:55.957653 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-222d64c6-530c-4a27-82c9-3a35f5f4202f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:56.014201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1324/2662] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:56.023835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:56.025247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:56.081324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0274d-d5eb-4e7b-abae-5987815c4539 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:56.085014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3683eb-d881-48a0-9ed6-6dc55f451ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:56.085659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3683eb-d881-48a0-9ed6-6dc55f451ccf service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:56.085945 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3683eb-d881-48a0-9ed6-6dc55f451ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:56.086226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3683eb-d881-48a0-9ed6-6dc55f451ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:56.095428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3683eb-d881-48a0-9ed6-6dc55f451ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:56.096865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3683eb-d881-48a0-9ed6-6dc55f451ccf service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:56.106159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1339/2663] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:56.374156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d64c6-530c-4a27-82c9-3a35f5f4202f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:56.375616 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-222d64c6-530c-4a27-82c9-3a35f5f4202f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:56.388296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d64c6-530c-4a27-82c9-3a35f5f4202f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:56.393825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:56.394703 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:56.394986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:56.395372 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] PATH_INFO: `/v3/projects/46395296793949c29ea77ffe0a8428db` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:56.396574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=46395296793949c29ea77ffe0a8428db)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:56.398546 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:56.445750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46395296793949c29ea77ffe0a8428db', 'request_id': 'req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:56.453241 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46395296793949c29ea77ffe0a8428db', 'request_id': 'req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:56.460092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46395296793949c29ea77ffe0a8428db', 'request_id': 'req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:56.470489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] The token cache is being invalidate because project 46395296793949c29ea77ffe0a8428db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:56.470796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:56.471431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:56.472952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1325/2664] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:56 2022] DELETE /identity/v3/projects/46395296793949c29ea77ffe0a8428db => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:56.476890 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3683eb-d881-48a0-9ed6-6dc55f451ccf service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:56.486536 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3683eb-d881-48a0-9ed6-6dc55f451ccf service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:56.544774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:56.553652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:56.554747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:56.555133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:56.555482 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] PATH_INFO: `/v3/users/60fbd86fe5b34828a7248e350d98fac8` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:56.556959 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=60fbd86fe5b34828a7248e350d98fac8)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:56.558885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:56.604491 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60fbd86fe5b34828a7248e350d98fac8'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:56.605139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60fbd86fe5b34828a7248e350d98fac8'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:56.605741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60fbd86fe5b34828a7248e350d98fac8'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:56.613076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60fbd86fe5b34828a7248e350d98fac8'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:56.620252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-9e646781-5dc5-47f6-8a32-92b6d399df1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60fbd86fe5b34828a7248e350d98fac8'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:56.637753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1340/2665] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:24:56 2022] DELETE /identity/v3/users/60fbd86fe5b34828a7248e350d98fac8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:56.902001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:56.903595 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:56.918927 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf6707-6fa5-4a45-b420-54c8f61a42dd admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:56.976447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:56.977300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:56.977592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:56.977875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] PATH_INFO: `/v3/projects/31dcfa6930ab4424a404cb18749d5521` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:56.979208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31dcfa6930ab4424a404cb18749d5521)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:56.981142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:56.987769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:57.027000 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31dcfa6930ab4424a404cb18749d5521', 'request_id': 'req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:57.034560 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31dcfa6930ab4424a404cb18749d5521', 'request_id': 'req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:57.040629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31dcfa6930ab4424a404cb18749d5521', 'request_id': 'req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:57.051641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] The token cache is being invalidate because project 31dcfa6930ab4424a404cb18749d5521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:24:57.051932 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:57.052561 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0fdfe0cf-6d4b-4849-b8e2-626b4b163568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:24:57.054146 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1326/2666] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:24:56 2022] DELETE /identity/v3/projects/31dcfa6930ab4424a404cb18749d5521 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:57.059147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0a464-7608-43ee-968d-3d92ccc12feb None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.059633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0a464-7608-43ee-968d-3d92ccc12feb None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.059743 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0a464-7608-43ee-968d-3d92ccc12feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.076901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5ed0a464-7608-43ee-968d-3d92ccc12feb None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:57.078209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5ed0a464-7608-43ee-968d-3d92ccc12feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:57.135594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1341/2667] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:57.143227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c9a21-ce42-4739-afb2-1157486b243e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.143518 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c9a21-ce42-4739-afb2-1157486b243e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.143801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882c9a21-ce42-4739-afb2-1157486b243e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.161919 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-882c9a21-ce42-4739-afb2-1157486b243e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:57.163128 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-882c9a21-ce42-4739-afb2-1157486b243e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:57.170925 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1327/2668] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:57.175761 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0a464-7608-43ee-968d-3d92ccc12feb None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.185433 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0a464-7608-43ee-968d-3d92ccc12feb None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.190110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e2ad-6418-4273-bf39-78997881805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.190763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e2ad-6418-4273-bf39-78997881805e admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.191074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e2ad-6418-4273-bf39-78997881805e admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.191374 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e2ad-6418-4273-bf39-78997881805e admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.192109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e2ad-6418-4273-bf39-78997881805e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:24:57.192424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e2ad-6418-4273-bf39-78997881805e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.194031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e2ad-6418-4273-bf39-78997881805e admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.205838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1342/2669] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:24:57.210123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c9a21-ce42-4739-afb2-1157486b243e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.219640 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882c9a21-ce42-4739-afb2-1157486b243e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.224797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38c44a-aede-4331-9683-360b2e62091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.225443 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38c44a-aede-4331-9683-360b2e62091f admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.225729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38c44a-aede-4331-9683-360b2e62091f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.226010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b38c44a-aede-4331-9683-360b2e62091f admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.226746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38c44a-aede-4331-9683-360b2e62091f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.228729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b38c44a-aede-4331-9683-360b2e62091f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.256344 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1328/2670] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:57 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:57.259695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e2ad-6418-4273-bf39-78997881805e admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.270845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e2ad-6418-4273-bf39-78997881805e admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.326063 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb59b6-4dac-4961-8ec5-dff9f2fd2913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.326727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb59b6-4dac-4961-8ec5-dff9f2fd2913 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.327014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb59b6-4dac-4961-8ec5-dff9f2fd2913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.327325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb59b6-4dac-4961-8ec5-dff9f2fd2913 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.328019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb59b6-4dac-4961-8ec5-dff9f2fd2913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.329940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb59b6-4dac-4961-8ec5-dff9f2fd2913 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.337821 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:57.351163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1343/2671] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:57 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:57.355371 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38c44a-aede-4331-9683-360b2e62091f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.365565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b38c44a-aede-4331-9683-360b2e62091f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.370124 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7674bdd-bbdd-462f-af62-1540ec2c51f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.370777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7674bdd-bbdd-462f-af62-1540ec2c51f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.371091 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7674bdd-bbdd-462f-af62-1540ec2c51f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.371404 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7674bdd-bbdd-462f-af62-1540ec2c51f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.372156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7674bdd-bbdd-462f-af62-1540ec2c51f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.373741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7674bdd-bbdd-462f-af62-1540ec2c51f6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.383870 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1329/2672] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:57.387489 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb59b6-4dac-4961-8ec5-dff9f2fd2913 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.397818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb59b6-4dac-4961-8ec5-dff9f2fd2913 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.402402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63f302-e4d3-4364-80a7-0ceab8ddf30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.403061 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63f302-e4d3-4364-80a7-0ceab8ddf30a admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.403365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63f302-e4d3-4364-80a7-0ceab8ddf30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.403645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63f302-e4d3-4364-80a7-0ceab8ddf30a admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.404369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63f302-e4d3-4364-80a7-0ceab8ddf30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.405986 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc63f302-e4d3-4364-80a7-0ceab8ddf30a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.418556 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1344/2673] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:57.422445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7674bdd-bbdd-462f-af62-1540ec2c51f6 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.432268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7674bdd-bbdd-462f-af62-1540ec2c51f6 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.436827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923c913-1fcf-4358-b229-56b5c7460bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.437690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c913-1fcf-4358-b229-56b5c7460bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.437972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c913-1fcf-4358-b229-56b5c7460bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.438254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923c913-1fcf-4358-b229-56b5c7460bb2 admin admin] PATH_INFO: `/v3/projects/387e3e504cdb4abba8aaae2ab32a8ddb/users/a8f124e18c644815bff0d95e52ad667f/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.452584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923c913-1fcf-4358-b229-56b5c7460bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387e3e504cdb4abba8aaae2ab32a8ddb, user_id=a8f124e18c644815bff0d95e52ad667f, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.455252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923c913-1fcf-4358-b229-56b5c7460bb2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.469061 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1330/2674] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:57 2022] PUT /identity/v3/projects/387e3e504cdb4abba8aaae2ab32a8ddb/users/a8f124e18c644815bff0d95e52ad667f/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:57.480342 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae1e04-cae7-4a34-86b8-0e8d7383afae None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.480581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae1e04-cae7-4a34-86b8-0e8d7383afae None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.480867 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae1e04-cae7-4a34-86b8-0e8d7383afae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.507383 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b2ae1e04-cae7-4a34-86b8-0e8d7383afae None None] MFA Rules not processed for user `a8f124e18c644815bff0d95e52ad667f`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:57.508619 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b2ae1e04-cae7-4a34-86b8-0e8d7383afae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:57.570379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1345/2675] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:57.574821 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923c913-1fcf-4358-b229-56b5c7460bb2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.635648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923c913-1fcf-4358-b229-56b5c7460bb2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.640220 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182dd3af-5fc7-4a42-8cfe-b84e4de2fd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.640879 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182dd3af-5fc7-4a42-8cfe-b84e4de2fd99 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.641169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182dd3af-5fc7-4a42-8cfe-b84e4de2fd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.641455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182dd3af-5fc7-4a42-8cfe-b84e4de2fd99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.642154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182dd3af-5fc7-4a42-8cfe-b84e4de2fd99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.644179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182dd3af-5fc7-4a42-8cfe-b84e4de2fd99 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.670444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1331/2676] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:24:57 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:57.674840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae1e04-cae7-4a34-86b8-0e8d7383afae None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.686270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae1e04-cae7-4a34-86b8-0e8d7383afae None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.740773 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8c732-d84d-44c5-8e3b-f8f1dc6860cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.741370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8c732-d84d-44c5-8e3b-f8f1dc6860cd admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.741656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8c732-d84d-44c5-8e3b-f8f1dc6860cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.741935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8c732-d84d-44c5-8e3b-f8f1dc6860cd admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.742610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8c732-d84d-44c5-8e3b-f8f1dc6860cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.744555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8c732-d84d-44c5-8e3b-f8f1dc6860cd admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.763865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1346/2677] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:24:57 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:24:57.767745 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182dd3af-5fc7-4a42-8cfe-b84e4de2fd99 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.778092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182dd3af-5fc7-4a42-8cfe-b84e4de2fd99 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.782665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84b2f1-3221-4768-9335-dfa60d7907fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.783360 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84b2f1-3221-4768-9335-dfa60d7907fd admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.784174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84b2f1-3221-4768-9335-dfa60d7907fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.784454 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84b2f1-3221-4768-9335-dfa60d7907fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.785179 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84b2f1-3221-4768-9335-dfa60d7907fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.786776 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84b2f1-3221-4768-9335-dfa60d7907fd admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.797611 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1332/2678] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:57.801905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8c732-d84d-44c5-8e3b-f8f1dc6860cd admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.811842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8c732-d84d-44c5-8e3b-f8f1dc6860cd admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.816462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c46a7d-c9c5-4865-84c1-8c436db14c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.817330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c46a7d-c9c5-4865-84c1-8c436db14c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.817620 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c46a7d-c9c5-4865-84c1-8c436db14c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.817900 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c46a7d-c9c5-4865-84c1-8c436db14c88 admin admin] PATH_INFO: `/v3/projects/e7638329af1e44f5bfea88816c039267/users/604d74a5fbc241f49cb67aa1628d6bd0/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.832530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c46a7d-c9c5-4865-84c1-8c436db14c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7638329af1e44f5bfea88816c039267, user_id=604d74a5fbc241f49cb67aa1628d6bd0, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.835303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c46a7d-c9c5-4865-84c1-8c436db14c88 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.849424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1347/2679] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:57 2022] PUT /identity/v3/projects/e7638329af1e44f5bfea88816c039267/users/604d74a5fbc241f49cb67aa1628d6bd0/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:57.853615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84b2f1-3221-4768-9335-dfa60d7907fd admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.859941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:57.864802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84b2f1-3221-4768-9335-dfa60d7907fd admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.925829 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b18b7b-8be5-4133-aee0-d9f498c8dea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.926499 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b18b7b-8be5-4133-aee0-d9f498c8dea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.926787 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b18b7b-8be5-4133-aee0-d9f498c8dea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.927090 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b18b7b-8be5-4133-aee0-d9f498c8dea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.927816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b18b7b-8be5-4133-aee0-d9f498c8dea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.929438 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b18b7b-8be5-4133-aee0-d9f498c8dea4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.938943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1333/2680] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:24:57.943032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c46a7d-c9c5-4865-84c1-8c436db14c88 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:57.953592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c46a7d-c9c5-4865-84c1-8c436db14c88 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:57.958262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa111d8-5fcd-4789-b309-d708a681702c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:57.959169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa111d8-5fcd-4789-b309-d708a681702c admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.959461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa111d8-5fcd-4789-b309-d708a681702c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.959741 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa111d8-5fcd-4789-b309-d708a681702c admin admin] PATH_INFO: `/v3/projects/e7638329af1e44f5bfea88816c039267/users/604d74a5fbc241f49cb67aa1628d6bd0/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:57.961749 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa111d8-5fcd-4789-b309-d708a681702c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7638329af1e44f5bfea88816c039267, user_id=604d74a5fbc241f49cb67aa1628d6bd0, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:57.964433 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa111d8-5fcd-4789-b309-d708a681702c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:57.975355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1348/2681] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:24:57 2022] PUT /identity/v3/projects/e7638329af1e44f5bfea88816c039267/users/604d74a5fbc241f49cb67aa1628d6bd0/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:24:57.985296 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34aecc-59b1-4565-a821-344cdd95385b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:57.985795 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34aecc-59b1-4565-a821-344cdd95385b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:57.985914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c34aecc-59b1-4565-a821-344cdd95385b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:58.014223 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7c34aecc-59b1-4565-a821-344cdd95385b None None] MFA Rules not processed for user `604d74a5fbc241f49cb67aa1628d6bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:24:58.015429 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7c34aecc-59b1-4565-a821-344cdd95385b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:58.084406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1334/2682] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:24:57 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:24:58.095858 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa111d8-5fcd-4789-b309-d708a681702c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:58.098165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7fa111d8-5fcd-4789-b309-d708a681702c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:24:58.124175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:24:58.155795 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa111d8-5fcd-4789-b309-d708a681702c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:58.160078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634f9d9-1342-430f-a4f7-3a701e578a11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:58.160717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634f9d9-1342-430f-a4f7-3a701e578a11 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:58.161003 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634f9d9-1342-430f-a4f7-3a701e578a11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:58.161280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8634f9d9-1342-430f-a4f7-3a701e578a11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:58.170204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634f9d9-1342-430f-a4f7-3a701e578a11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:58.171690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8634f9d9-1342-430f-a4f7-3a701e578a11 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:58.181091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1349/2683] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:58 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:24:58.249856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34aecc-59b1-4565-a821-344cdd95385b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:24:58.260152 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c34aecc-59b1-4565-a821-344cdd95385b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:24:58.263961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df286233-4282-497c-a778-03b0cc33675c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:24:58.264462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286233-4282-497c-a778-03b0cc33675c service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:24:58.264803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286233-4282-497c-a778-03b0cc33675c service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:24:58.265006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286233-4282-497c-a778-03b0cc33675c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:24:58.324816 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df286233-4282-497c-a778-03b0cc33675c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:24:58.326262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df286233-4282-497c-a778-03b0cc33675c service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:24:58.335756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1335/2684] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:24:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:02.055254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634f9d9-1342-430f-a4f7-3a701e578a11 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:02.056763 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8634f9d9-1342-430f-a4f7-3a701e578a11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:02.119597 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8634f9d9-1342-430f-a4f7-3a701e578a11 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:02.124062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:02.124943 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:02.125227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:02.125502 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] PATH_INFO: `/v3/users/371e853f9ae3481dbd34a720803526c8` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:02.126655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371e853f9ae3481dbd34a720803526c8)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:02.128592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:02.173733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371e853f9ae3481dbd34a720803526c8'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.174405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371e853f9ae3481dbd34a720803526c8'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.175006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371e853f9ae3481dbd34a720803526c8'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.181805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371e853f9ae3481dbd34a720803526c8'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.188826 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371e853f9ae3481dbd34a720803526c8'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.211403 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1350/2685] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:02 2022] DELETE /identity/v3/users/371e853f9ae3481dbd34a720803526c8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:02.215817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094835f8-1864-417e-83ff-546c9d65e98b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:02.216313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094835f8-1864-417e-83ff-546c9d65e98b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:02.216413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094835f8-1864-417e-83ff-546c9d65e98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:02.234520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-094835f8-1864-417e-83ff-546c9d65e98b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:02.235773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-094835f8-1864-417e-83ff-546c9d65e98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:02.254341 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:02.294872 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1336/2686] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:02.304441 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:02.305819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:02.353753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:02.365906 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0a9255-584b-43d3-8d39-8b554a3ea130 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:02.369564 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4d41a-74ea-4879-8c95-0187432ed8b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:02.370212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4d41a-74ea-4879-8c95-0187432ed8b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:02.370501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4d41a-74ea-4879-8c95-0187432ed8b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:02.370780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c4d41a-74ea-4879-8c95-0187432ed8b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:02.380258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4d41a-74ea-4879-8c95-0187432ed8b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:02.381701 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c4d41a-74ea-4879-8c95-0187432ed8b5 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:02.391069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1351/2687] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:02.652462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094835f8-1864-417e-83ff-546c9d65e98b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:02.653978 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-094835f8-1864-417e-83ff-546c9d65e98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:02.667509 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094835f8-1864-417e-83ff-546c9d65e98b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:02.672209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:02.673078 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:02.673362 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:02.673641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] PATH_INFO: `/v3/projects/4ef7c390e8014ca9906e82eef7fd387a` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:02.675414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef7c390e8014ca9906e82eef7fd387a)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:02.677530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:02.724189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef7c390e8014ca9906e82eef7fd387a', 'request_id': 'req-f8ff6821-9c98-416c-8621-5fecd82fa86b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.731144 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef7c390e8014ca9906e82eef7fd387a', 'request_id': 'req-f8ff6821-9c98-416c-8621-5fecd82fa86b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.737619 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef7c390e8014ca9906e82eef7fd387a', 'request_id': 'req-f8ff6821-9c98-416c-8621-5fecd82fa86b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.747743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] The token cache is being invalidate because project 4ef7c390e8014ca9906e82eef7fd387a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:02.748028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.748650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.750214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1337/2688] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:02 2022] DELETE /identity/v3/projects/4ef7c390e8014ca9906e82eef7fd387a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:02.754769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4d41a-74ea-4879-8c95-0187432ed8b5 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:02.764487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c4d41a-74ea-4879-8c95-0187432ed8b5 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:02.818885 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:02.819772 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:02.820055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:02.820329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] PATH_INFO: `/v3/users/4b7ce8bef3284dcea7edfcfedba0b51e` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:02.821500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b7ce8bef3284dcea7edfcfedba0b51e)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:02.823450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:02.867358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b7ce8bef3284dcea7edfcfedba0b51e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.867994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b7ce8bef3284dcea7edfcfedba0b51e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.868617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b7ce8bef3284dcea7edfcfedba0b51e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.875488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b7ce8bef3284dcea7edfcfedba0b51e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.883035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bcd415ad-a360-49ce-8abf-df3826985095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b7ce8bef3284dcea7edfcfedba0b51e'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:02.899912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1352/2689] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:02 2022] DELETE /identity/v3/users/4b7ce8bef3284dcea7edfcfedba0b51e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:03.167699 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:03.169166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:03.183674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff6821-9c98-416c-8621-5fecd82fa86b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:03.246926 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:03.247731 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.248010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.248283 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] PATH_INFO: `/v3/projects/4f9a7f3a29c74c46ac39df8b525ae7db` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.249446 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f9a7f3a29c74c46ac39df8b525ae7db)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:03.251366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:03.296208 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f9a7f3a29c74c46ac39df8b525ae7db', 'request_id': 'req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:03.297784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:03.303660 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f9a7f3a29c74c46ac39df8b525ae7db', 'request_id': 'req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:03.310772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f9a7f3a29c74c46ac39df8b525ae7db', 'request_id': 'req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:03.321641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] The token cache is being invalidate because project 4f9a7f3a29c74c46ac39df8b525ae7db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:03.321933 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:03.322603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-e32a554d-1d71-4e45-8cb1-e4a80a8849a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:03.324212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1338/2690] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:03 2022] DELETE /identity/v3/projects/4f9a7f3a29c74c46ac39df8b525ae7db => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:03.329030 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bb694-f2df-46c8-a7de-cdb451e7dc62 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.329480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bb694-f2df-46c8-a7de-cdb451e7dc62 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.329599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283bb694-f2df-46c8-a7de-cdb451e7dc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.348643 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-283bb694-f2df-46c8-a7de-cdb451e7dc62 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:03.349852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-283bb694-f2df-46c8-a7de-cdb451e7dc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:03.414425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1353/2691] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:03.422288 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ab78-34ca-41b1-8a08-84c5c8e99a51 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.422582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ab78-34ca-41b1-8a08-84c5c8e99a51 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.422865 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81ab78-34ca-41b1-8a08-84c5c8e99a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.441603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-7c81ab78-34ca-41b1-8a08-84c5c8e99a51 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:03.442860 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-7c81ab78-34ca-41b1-8a08-84c5c8e99a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:03.450856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1339/2692] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:03.455149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bb694-f2df-46c8-a7de-cdb451e7dc62 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:03.465412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283bb694-f2df-46c8-a7de-cdb451e7dc62 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:03.470322 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d90a0d-283b-42bf-95bc-e315092fe107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:03.470974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d90a0d-283b-42bf-95bc-e315092fe107 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.471888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d90a0d-283b-42bf-95bc-e315092fe107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.472220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d90a0d-283b-42bf-95bc-e315092fe107 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.472980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d90a0d-283b-42bf-95bc-e315092fe107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:25:03.473311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d90a0d-283b-42bf-95bc-e315092fe107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:03.474915 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d90a0d-283b-42bf-95bc-e315092fe107 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:03.476425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:03.489117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1354/2693] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:25:03.493377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81ab78-34ca-41b1-8a08-84c5c8e99a51 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:03.502734 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81ab78-34ca-41b1-8a08-84c5c8e99a51 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:03.507554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44366a-b77a-4af1-97d1-da4ad3b14417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:03.508207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44366a-b77a-4af1-97d1-da4ad3b14417 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.508494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44366a-b77a-4af1-97d1-da4ad3b14417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.508772 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44366a-b77a-4af1-97d1-da4ad3b14417 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.509460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44366a-b77a-4af1-97d1-da4ad3b14417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:03.511419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44366a-b77a-4af1-97d1-da4ad3b14417 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:03.537449 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1340/2694] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:03 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:03.541329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d90a0d-283b-42bf-95bc-e315092fe107 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:03.551819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d90a0d-283b-42bf-95bc-e315092fe107 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:03.609348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b4e36-baf9-45f0-a596-378e882aea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:03.610020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b4e36-baf9-45f0-a596-378e882aea55 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.610310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b4e36-baf9-45f0-a596-378e882aea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.610593 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b4e36-baf9-45f0-a596-378e882aea55 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.611316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b4e36-baf9-45f0-a596-378e882aea55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:03.613261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593b4e36-baf9-45f0-a596-378e882aea55 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:03.633077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1355/2695] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:03 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:03.637212 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44366a-b77a-4af1-97d1-da4ad3b14417 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:03.647845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44366a-b77a-4af1-97d1-da4ad3b14417 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:03.652505 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9956112-dbd5-4e9f-9540-7a81bb3b2819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:03.653163 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9956112-dbd5-4e9f-9540-7a81bb3b2819 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.653448 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9956112-dbd5-4e9f-9540-7a81bb3b2819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.653724 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9956112-dbd5-4e9f-9540-7a81bb3b2819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.654445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9956112-dbd5-4e9f-9540-7a81bb3b2819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:03.656076 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9956112-dbd5-4e9f-9540-7a81bb3b2819 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:03.667657 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1341/2696] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:03.671379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b4e36-baf9-45f0-a596-378e882aea55 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:03.681504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b4e36-baf9-45f0-a596-378e882aea55 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:03.686174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3347b02-429b-47ad-ada2-81e71c9fd74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:03.686839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347b02-429b-47ad-ada2-81e71c9fd74a admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.687164 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347b02-429b-47ad-ada2-81e71c9fd74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.687451 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3347b02-429b-47ad-ada2-81e71c9fd74a admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.688276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3347b02-429b-47ad-ada2-81e71c9fd74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:03.690436 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3347b02-429b-47ad-ada2-81e71c9fd74a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:03.700520 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1356/2697] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:03.705463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9956112-dbd5-4e9f-9540-7a81bb3b2819 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:03.717591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9956112-dbd5-4e9f-9540-7a81bb3b2819 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:03.722484 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82e42f-37eb-43ad-a163-ba96604119db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:03.723263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82e42f-37eb-43ad-a163-ba96604119db admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.723554 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82e42f-37eb-43ad-a163-ba96604119db admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.723838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82e42f-37eb-43ad-a163-ba96604119db admin admin] PATH_INFO: `/v3/projects/61007346e3a944c7a804a6595b5eb0f3/users/f817b281693d48468e25d327917e9288/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.738176 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82e42f-37eb-43ad-a163-ba96604119db admin admin] RBAC: Authorizing `identity:create_grant(project_id=61007346e3a944c7a804a6595b5eb0f3, user_id=f817b281693d48468e25d327917e9288, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:03.740895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da82e42f-37eb-43ad-a163-ba96604119db admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:03.754265 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1342/2698] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:03 2022] PUT /identity/v3/projects/61007346e3a944c7a804a6595b5eb0f3/users/f817b281693d48468e25d327917e9288/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:03.764422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b88b2-2687-4f32-8c50-85eec90991e2 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.764720 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b88b2-2687-4f32-8c50-85eec90991e2 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.765000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6b88b2-2687-4f32-8c50-85eec90991e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.791066 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-8d6b88b2-2687-4f32-8c50-85eec90991e2 None None] MFA Rules not processed for user `f817b281693d48468e25d327917e9288`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:03.792270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8d6b88b2-2687-4f32-8c50-85eec90991e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:03.880674 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1357/2699] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:03.885262 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82e42f-37eb-43ad-a163-ba96604119db admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:03.946211 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82e42f-37eb-43ad-a163-ba96604119db admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:03.950975 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd5ade-220a-4a6a-94b0-735af78f4641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:03.951659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd5ade-220a-4a6a-94b0-735af78f4641 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:03.951942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd5ade-220a-4a6a-94b0-735af78f4641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:03.952217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd5ade-220a-4a6a-94b0-735af78f4641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:03.952902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd5ade-220a-4a6a-94b0-735af78f4641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:03.954811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd5ade-220a-4a6a-94b0-735af78f4641 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:03.980574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1343/2700] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:03 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:03.984541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b88b2-2687-4f32-8c50-85eec90991e2 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:03.995358 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6b88b2-2687-4f32-8c50-85eec90991e2 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:04.051468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0c191-6255-4862-b5d8-d20e3e42f38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:04.052064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0c191-6255-4862-b5d8-d20e3e42f38c admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:04.052355 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0c191-6255-4862-b5d8-d20e3e42f38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:04.052635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0c191-6255-4862-b5d8-d20e3e42f38c admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:04.053465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0c191-6255-4862-b5d8-d20e3e42f38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:04.055449 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0c191-6255-4862-b5d8-d20e3e42f38c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:04.063180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:04.076064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1358/2701] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:03 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:04.079482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd5ade-220a-4a6a-94b0-735af78f4641 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:04.090226 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd5ade-220a-4a6a-94b0-735af78f4641 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:04.095082 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a7e1e-a625-451a-85e0-737fb94b355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:04.095738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a7e1e-a625-451a-85e0-737fb94b355f admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:04.096022 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a7e1e-a625-451a-85e0-737fb94b355f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:04.096895 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a7e1e-a625-451a-85e0-737fb94b355f admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:04.097615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a7e1e-a625-451a-85e0-737fb94b355f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:04.099238 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a7e1e-a625-451a-85e0-737fb94b355f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:04.110508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1344/2702] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:04.114705 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0c191-6255-4862-b5d8-d20e3e42f38c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:04.124824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0c191-6255-4862-b5d8-d20e3e42f38c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:04.129527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c9457-93f1-46fe-a057-61f37b21990d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:04.130404 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c9457-93f1-46fe-a057-61f37b21990d admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:04.130693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c9457-93f1-46fe-a057-61f37b21990d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:04.130975 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3c9457-93f1-46fe-a057-61f37b21990d admin admin] PATH_INFO: `/v3/projects/80cf730c3b7940899654a8c9f8c57248/users/bc5f5bda72d54b689c7723c256426833/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:04.146006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c9457-93f1-46fe-a057-61f37b21990d admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cf730c3b7940899654a8c9f8c57248, user_id=bc5f5bda72d54b689c7723c256426833, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:04.148798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3c9457-93f1-46fe-a057-61f37b21990d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:04.163325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1359/2703] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:04 2022] PUT /identity/v3/projects/80cf730c3b7940899654a8c9f8c57248/users/bc5f5bda72d54b689c7723c256426833/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:04.167592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a7e1e-a625-451a-85e0-737fb94b355f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:04.177690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a7e1e-a625-451a-85e0-737fb94b355f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:04.225117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:04.237517 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68836440-2437-4e37-85ee-dfcb66283584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:04.238188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68836440-2437-4e37-85ee-dfcb66283584 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:04.238472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68836440-2437-4e37-85ee-dfcb66283584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:04.238750 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68836440-2437-4e37-85ee-dfcb66283584 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:04.239516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68836440-2437-4e37-85ee-dfcb66283584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:04.241100 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68836440-2437-4e37-85ee-dfcb66283584 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:04.250723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1345/2704] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:25:04.254933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c9457-93f1-46fe-a057-61f37b21990d admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:04.265382 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3c9457-93f1-46fe-a057-61f37b21990d admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:04.270745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4aedc-18b7-4d52-8ca6-44ecd5869b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:04.271664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4aedc-18b7-4d52-8ca6-44ecd5869b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:04.271949 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4aedc-18b7-4d52-8ca6-44ecd5869b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:04.272229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4aedc-18b7-4d52-8ca6-44ecd5869b50 admin admin] PATH_INFO: `/v3/projects/80cf730c3b7940899654a8c9f8c57248/users/bc5f5bda72d54b689c7723c256426833/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:04.274198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4aedc-18b7-4d52-8ca6-44ecd5869b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cf730c3b7940899654a8c9f8c57248, user_id=bc5f5bda72d54b689c7723c256426833, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:04.276901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4aedc-18b7-4d52-8ca6-44ecd5869b50 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:04.287655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1360/2705] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:04 2022] PUT /identity/v3/projects/80cf730c3b7940899654a8c9f8c57248/users/bc5f5bda72d54b689c7723c256426833/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:04.298432 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc4aaf-9ef8-4164-9882-e66adf161e59 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:04.298728 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc4aaf-9ef8-4164-9882-e66adf161e59 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:04.299010 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc4aaf-9ef8-4164-9882-e66adf161e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:04.324635 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d2bc4aaf-9ef8-4164-9882-e66adf161e59 None None] MFA Rules not processed for user `bc5f5bda72d54b689c7723c256426833`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:04.325774 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d2bc4aaf-9ef8-4164-9882-e66adf161e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:04.395501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1346/2706] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:04.406812 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4aedc-18b7-4d52-8ca6-44ecd5869b50 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:04.409203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-1ac4aedc-18b7-4d52-8ca6-44ecd5869b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:04.464468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4aedc-18b7-4d52-8ca6-44ecd5869b50 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:04.468345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b01d4-370d-407d-a10b-bcaae0b302e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:04.468978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b01d4-370d-407d-a10b-bcaae0b302e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:04.469267 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b01d4-370d-407d-a10b-bcaae0b302e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:04.469544 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b01d4-370d-407d-a10b-bcaae0b302e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:04.478360 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b01d4-370d-407d-a10b-bcaae0b302e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:04.479830 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b01d4-370d-407d-a10b-bcaae0b302e9 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:04.489274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1361/2707] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:04.516455 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc4aaf-9ef8-4164-9882-e66adf161e59 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:04.577729 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc4aaf-9ef8-4164-9882-e66adf161e59 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:04.582776 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:04.583672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:04.583957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:04.584235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] PATH_INFO: `/v3/users/f817b281693d48468e25d327917e9288` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:04.585421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f817b281693d48468e25d327917e9288)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:04.587366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:04.633634 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f817b281693d48468e25d327917e9288'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:04.634314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f817b281693d48468e25d327917e9288'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:04.634985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f817b281693d48468e25d327917e9288'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:04.645066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f817b281693d48468e25d327917e9288'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:04.652271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f817b281693d48468e25d327917e9288'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:04.659520 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:04.671339 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1347/2708] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:04 2022] DELETE /identity/v3/users/f817b281693d48468e25d327917e9288 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:04.675672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fdccb-2239-4f9b-84a2-5d7f2296ace3 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:04.675964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fdccb-2239-4f9b-84a2-5d7f2296ace3 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:04.676244 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257fdccb-2239-4f9b-84a2-5d7f2296ace3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:04.694202 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-257fdccb-2239-4f9b-84a2-5d7f2296ace3 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:04.695458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-257fdccb-2239-4f9b-84a2-5d7f2296ace3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:04.751345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1362/2709] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:04.761480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:04.762888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:04.820434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d1a77-c5e7-4b0d-a8c2-dac766c6234d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:04.824052 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05b231-49ca-4c0b-8e87-bef6984ce769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:04.824696 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05b231-49ca-4c0b-8e87-bef6984ce769 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:04.824983 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05b231-49ca-4c0b-8e87-bef6984ce769 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:04.825263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05b231-49ca-4c0b-8e87-bef6984ce769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:04.834256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05b231-49ca-4c0b-8e87-bef6984ce769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:04.835661 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05b231-49ca-4c0b-8e87-bef6984ce769 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:04.845389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1348/2710] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:05.115868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fdccb-2239-4f9b-84a2-5d7f2296ace3 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:05.117299 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-257fdccb-2239-4f9b-84a2-5d7f2296ace3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:05.130475 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257fdccb-2239-4f9b-84a2-5d7f2296ace3 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:05.135266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:05.136134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:05.136450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:05.136737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] PATH_INFO: `/v3/projects/61007346e3a944c7a804a6595b5eb0f3` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:05.137960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61007346e3a944c7a804a6595b5eb0f3)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:05.139934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:05.187304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61007346e3a944c7a804a6595b5eb0f3', 'request_id': 'req-fd92f8c0-4d94-4db1-a9ed-681012db6315'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.189732 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:05.194675 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61007346e3a944c7a804a6595b5eb0f3', 'request_id': 'req-fd92f8c0-4d94-4db1-a9ed-681012db6315'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.201461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61007346e3a944c7a804a6595b5eb0f3', 'request_id': 'req-fd92f8c0-4d94-4db1-a9ed-681012db6315'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.212064 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] The token cache is being invalidate because project 61007346e3a944c7a804a6595b5eb0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:05.212353 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.212988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.214611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1363/2711] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:05 2022] DELETE /identity/v3/projects/61007346e3a944c7a804a6595b5eb0f3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:05.218754 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05b231-49ca-4c0b-8e87-bef6984ce769 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:05.228811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05b231-49ca-4c0b-8e87-bef6984ce769 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:05.284026 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:05.284868 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:05.285150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:05.285428 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] PATH_INFO: `/v3/users/bc5f5bda72d54b689c7723c256426833` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:05.286599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc5f5bda72d54b689c7723c256426833)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:05.288535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:05.332234 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc5f5bda72d54b689c7723c256426833'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.332863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc5f5bda72d54b689c7723c256426833'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.333453 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc5f5bda72d54b689c7723c256426833'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.340294 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc5f5bda72d54b689c7723c256426833'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.347841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bba51656-471a-41b8-b4a0-ec114fd8573f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc5f5bda72d54b689c7723c256426833'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.365585 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1349/2712] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:05 2022] DELETE /identity/v3/users/bc5f5bda72d54b689c7723c256426833 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:05.623707 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:05.625204 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:05.638162 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92f8c0-4d94-4db1-a9ed-681012db6315 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:05.695005 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:05.695900 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:05.696190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:05.696470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] PATH_INFO: `/v3/projects/80cf730c3b7940899654a8c9f8c57248` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:05.697651 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80cf730c3b7940899654a8c9f8c57248)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:05.699608 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:05.746389 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80cf730c3b7940899654a8c9f8c57248', 'request_id': 'req-d36af2e4-dd71-470f-b054-90d38fdbd5f0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.760336 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80cf730c3b7940899654a8c9f8c57248', 'request_id': 'req-d36af2e4-dd71-470f-b054-90d38fdbd5f0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.765549 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80cf730c3b7940899654a8c9f8c57248', 'request_id': 'req-d36af2e4-dd71-470f-b054-90d38fdbd5f0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.776989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] The token cache is being invalidate because project 80cf730c3b7940899654a8c9f8c57248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:05.777281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.777918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d36af2e4-dd71-470f-b054-90d38fdbd5f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:05.779498 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1364/2713] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:05 2022] DELETE /identity/v3/projects/80cf730c3b7940899654a8c9f8c57248 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:05.785015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954494f-8f77-470e-8657-611afa6d3c16 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:05.785311 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954494f-8f77-470e-8657-611afa6d3c16 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:05.785629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a954494f-8f77-470e-8657-611afa6d3c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:05.804102 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a954494f-8f77-470e-8657-611afa6d3c16 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:05.805314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a954494f-8f77-470e-8657-611afa6d3c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:05.810637 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:05.864709 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1350/2714] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:05.872366 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52caad-7170-4891-bbea-b7adc08db979 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:05.872668 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52caad-7170-4891-bbea-b7adc08db979 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:05.872957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52caad-7170-4891-bbea-b7adc08db979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:05.890824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0d52caad-7170-4891-bbea-b7adc08db979 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:05.892076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0d52caad-7170-4891-bbea-b7adc08db979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:05.900471 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1365/2715] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:05.904409 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954494f-8f77-470e-8657-611afa6d3c16 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:05.914568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a954494f-8f77-470e-8657-611afa6d3c16 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:05.919139 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa596584-570a-4b8b-a2f6-3c0c04a53e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:05.919785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa596584-570a-4b8b-a2f6-3c0c04a53e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:05.920072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa596584-570a-4b8b-a2f6-3c0c04a53e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:05.920355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa596584-570a-4b8b-a2f6-3c0c04a53e25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:05.921081 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa596584-570a-4b8b-a2f6-3c0c04a53e25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:25:05.921390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa596584-570a-4b8b-a2f6-3c0c04a53e25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:05.923013 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa596584-570a-4b8b-a2f6-3c0c04a53e25 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:05.934414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1351/2716] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:25:05.938688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52caad-7170-4891-bbea-b7adc08db979 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:05.948606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52caad-7170-4891-bbea-b7adc08db979 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:05.953393 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa460715-27a9-4e83-98cc-c4f8c38fb339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:05.954047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa460715-27a9-4e83-98cc-c4f8c38fb339 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:05.954335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa460715-27a9-4e83-98cc-c4f8c38fb339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:05.954616 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa460715-27a9-4e83-98cc-c4f8c38fb339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:05.955361 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa460715-27a9-4e83-98cc-c4f8c38fb339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:05.957854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa460715-27a9-4e83-98cc-c4f8c38fb339 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:05.982433 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1366/2717] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:05 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:05.986613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa596584-570a-4b8b-a2f6-3c0c04a53e25 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:05.996374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa596584-570a-4b8b-a2f6-3c0c04a53e25 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.054018 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebbdaa-6d2d-4529-b78a-0fe227be4623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.055134 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebbdaa-6d2d-4529-b78a-0fe227be4623 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.055435 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebbdaa-6d2d-4529-b78a-0fe227be4623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.055803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebbdaa-6d2d-4529-b78a-0fe227be4623 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.056701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebbdaa-6d2d-4529-b78a-0fe227be4623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.058896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebbdaa-6d2d-4529-b78a-0fe227be4623 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.077981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1352/2718] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:05 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:06.082675 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa460715-27a9-4e83-98cc-c4f8c38fb339 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:06.093270 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa460715-27a9-4e83-98cc-c4f8c38fb339 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.097988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd3571-1d28-4302-992e-e943db2f60a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.098666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd3571-1d28-4302-992e-e943db2f60a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.098963 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd3571-1d28-4302-992e-e943db2f60a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.099283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd3571-1d28-4302-992e-e943db2f60a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.100050 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd3571-1d28-4302-992e-e943db2f60a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.101660 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd3571-1d28-4302-992e-e943db2f60a5 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.113067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1367/2719] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:06.116673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebbdaa-6d2d-4529-b78a-0fe227be4623 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:06.126434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebbdaa-6d2d-4529-b78a-0fe227be4623 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.131543 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5608b9-1cdf-41fe-af8e-8772e71a6173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.132200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5608b9-1cdf-41fe-af8e-8772e71a6173 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.132489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5608b9-1cdf-41fe-af8e-8772e71a6173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.132769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5608b9-1cdf-41fe-af8e-8772e71a6173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.133539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5608b9-1cdf-41fe-af8e-8772e71a6173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.135150 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5608b9-1cdf-41fe-af8e-8772e71a6173 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.145468 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1353/2720] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:06.149382 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd3571-1d28-4302-992e-e943db2f60a5 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:06.159203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd3571-1d28-4302-992e-e943db2f60a5 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.163894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59fb7-4d39-4be0-a6a9-b57a1c23f173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.164769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59fb7-4d39-4be0-a6a9-b57a1c23f173 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.165310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59fb7-4d39-4be0-a6a9-b57a1c23f173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.165635 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b59fb7-4d39-4be0-a6a9-b57a1c23f173 admin admin] PATH_INFO: `/v3/projects/3a74db7002a74155b9d0a7f2791109fc/users/9db349ff9e0b4a92b7363313e2eb8016/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.180017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b59fb7-4d39-4be0-a6a9-b57a1c23f173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a74db7002a74155b9d0a7f2791109fc, user_id=9db349ff9e0b4a92b7363313e2eb8016, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.182662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b59fb7-4d39-4be0-a6a9-b57a1c23f173 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.196176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1368/2721] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:06 2022] PUT /identity/v3/projects/3a74db7002a74155b9d0a7f2791109fc/users/9db349ff9e0b4a92b7363313e2eb8016/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:06.206972 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7e2b1-d477-4531-acea-f2d7fdaaf2f9 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.207303 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7e2b1-d477-4531-acea-f2d7fdaaf2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.207589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7e2b1-d477-4531-acea-f2d7fdaaf2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.233269 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4cd7e2b1-d477-4531-acea-f2d7fdaaf2f9 None None] MFA Rules not processed for user `9db349ff9e0b4a92b7363313e2eb8016`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:06.234494 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4cd7e2b1-d477-4531-acea-f2d7fdaaf2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:06.297649 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1354/2722] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:06.302538 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59fb7-4d39-4be0-a6a9-b57a1c23f173 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:06.311292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:06.367439 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b59fb7-4d39-4be0-a6a9-b57a1c23f173 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.372151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db795f9-8e00-4157-9419-499f60408ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.372866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db795f9-8e00-4157-9419-499f60408ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.373157 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db795f9-8e00-4157-9419-499f60408ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.373440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db795f9-8e00-4157-9419-499f60408ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.374179 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db795f9-8e00-4157-9419-499f60408ad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.376145 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db795f9-8e00-4157-9419-499f60408ad0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.401076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1369/2723] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:06 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:06.405079 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7e2b1-d477-4531-acea-f2d7fdaaf2f9 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:06.415695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7e2b1-d477-4531-acea-f2d7fdaaf2f9 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.419227 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:06.472803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286d20f-bc51-4d0f-8b21-b42c1f30bba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.473471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286d20f-bc51-4d0f-8b21-b42c1f30bba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.473760 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286d20f-bc51-4d0f-8b21-b42c1f30bba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.474045 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286d20f-bc51-4d0f-8b21-b42c1f30bba1 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.474726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286d20f-bc51-4d0f-8b21-b42c1f30bba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.476782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286d20f-bc51-4d0f-8b21-b42c1f30bba1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.496021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1355/2724] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:06 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:06.499956 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db795f9-8e00-4157-9419-499f60408ad0 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:06.511406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db795f9-8e00-4157-9419-499f60408ad0 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.516846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe887d8-f8ba-4081-b042-4c7c00719d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.517513 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe887d8-f8ba-4081-b042-4c7c00719d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.517846 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe887d8-f8ba-4081-b042-4c7c00719d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.518225 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe887d8-f8ba-4081-b042-4c7c00719d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.518895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe887d8-f8ba-4081-b042-4c7c00719d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.520534 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe887d8-f8ba-4081-b042-4c7c00719d7f admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.531726 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1370/2725] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:06.535948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286d20f-bc51-4d0f-8b21-b42c1f30bba1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:06.545801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286d20f-bc51-4d0f-8b21-b42c1f30bba1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.550961 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05dc4b-dc0a-4c92-952b-9ff850e36645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.551869 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05dc4b-dc0a-4c92-952b-9ff850e36645 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.552159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05dc4b-dc0a-4c92-952b-9ff850e36645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.552441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05dc4b-dc0a-4c92-952b-9ff850e36645 admin admin] PATH_INFO: `/v3/projects/4089410a0540402c816b8b884c2587f1/users/1f4b48df06ca445caf9e447dbd79003c/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.567692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05dc4b-dc0a-4c92-952b-9ff850e36645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4089410a0540402c816b8b884c2587f1, user_id=1f4b48df06ca445caf9e447dbd79003c, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.570390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05dc4b-dc0a-4c92-952b-9ff850e36645 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.582626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1356/2726] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:06 2022] PUT /identity/v3/projects/4089410a0540402c816b8b884c2587f1/users/1f4b48df06ca445caf9e447dbd79003c/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:06.586421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe887d8-f8ba-4081-b042-4c7c00719d7f admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:06.596591 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe887d8-f8ba-4081-b042-4c7c00719d7f admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.651426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67d8f0-6ec0-4a1a-b40c-c693ef981144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.652024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67d8f0-6ec0-4a1a-b40c-c693ef981144 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.652295 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67d8f0-6ec0-4a1a-b40c-c693ef981144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.652574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67d8f0-6ec0-4a1a-b40c-c693ef981144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.653294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67d8f0-6ec0-4a1a-b40c-c693ef981144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.654895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67d8f0-6ec0-4a1a-b40c-c693ef981144 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.665117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1371/2727] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:06.669474 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05dc4b-dc0a-4c92-952b-9ff850e36645 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:06.680363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05dc4b-dc0a-4c92-952b-9ff850e36645 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.685153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724419b-e796-4976-8bc0-67385ad92a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.686034 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724419b-e796-4976-8bc0-67385ad92a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.686336 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724419b-e796-4976-8bc0-67385ad92a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.686612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724419b-e796-4976-8bc0-67385ad92a99 admin admin] PATH_INFO: `/v3/projects/4089410a0540402c816b8b884c2587f1/users/1f4b48df06ca445caf9e447dbd79003c/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.688656 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724419b-e796-4976-8bc0-67385ad92a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4089410a0540402c816b8b884c2587f1, user_id=1f4b48df06ca445caf9e447dbd79003c, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.691383 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724419b-e796-4976-8bc0-67385ad92a99 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.701513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1357/2728] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:06 2022] PUT /identity/v3/projects/4089410a0540402c816b8b884c2587f1/users/1f4b48df06ca445caf9e447dbd79003c/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:06.712606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f58ff6-590d-4664-be84-4b7f41ac0390 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.712905 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f58ff6-590d-4664-be84-4b7f41ac0390 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.713222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f58ff6-590d-4664-be84-4b7f41ac0390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.740028 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-b7f58ff6-590d-4664-be84-4b7f41ac0390 None None] MFA Rules not processed for user `1f4b48df06ca445caf9e447dbd79003c`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:06.741274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b7f58ff6-590d-4664-be84-4b7f41ac0390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:06.814062 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1372/2729] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:06.826396 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724419b-e796-4976-8bc0-67385ad92a99 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:06.828143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-1724419b-e796-4976-8bc0-67385ad92a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:06.885822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724419b-e796-4976-8bc0-67385ad92a99 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:06.889969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3374f7-1754-4c81-ae70-8116f7e33538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:06.890629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3374f7-1754-4c81-ae70-8116f7e33538 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:06.890941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3374f7-1754-4c81-ae70-8116f7e33538 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:06.891254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3374f7-1754-4c81-ae70-8116f7e33538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:06.900687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3374f7-1754-4c81-ae70-8116f7e33538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:06.902216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3374f7-1754-4c81-ae70-8116f7e33538 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:06.912148 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1358/2730] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:07.210825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f58ff6-590d-4664-be84-4b7f41ac0390 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:07.215242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:07.221688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f58ff6-590d-4664-be84-4b7f41ac0390 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:07.226690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31b2e3-cd0d-410a-a336-dfe77cdafc1a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b48df06ca445caf9e447dbd79003c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4089410a0540402c816b8b884c2587f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:07.227388 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31b2e3-cd0d-410a-a336-dfe77cdafc1a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:07.227683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31b2e3-cd0d-410a-a336-dfe77cdafc1a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:07.227965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f31b2e3-cd0d-410a-a336-dfe77cdafc1a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:07.228693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31b2e3-cd0d-410a-a336-dfe77cdafc1a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:25:07.229026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31b2e3-cd0d-410a-a336-dfe77cdafc1a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:07.230622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f31b2e3-cd0d-410a-a336-dfe77cdafc1a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:07.242778 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1373/2731] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:25:07.246664 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3374f7-1754-4c81-ae70-8116f7e33538 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:07.256203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3374f7-1754-4c81-ae70-8116f7e33538 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:07.260718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3f6ed-e454-49da-be84-0956497a7b8e tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b48df06ca445caf9e447dbd79003c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4089410a0540402c816b8b884c2587f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:07.261356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3f6ed-e454-49da-be84-0956497a7b8e tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:07.261643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3f6ed-e454-49da-be84-0956497a7b8e tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:07.261932 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3f6ed-e454-49da-be84-0956497a7b8e tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:07.262676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3f6ed-e454-49da-be84-0956497a7b8e tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:07.264603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3f6ed-e454-49da-be84-0956497a7b8e tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:07.289281 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1359/2732] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:07 2022] POST /identity/v3/projects => generated 353 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:07.293113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31b2e3-cd0d-410a-a336-dfe77cdafc1a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:07.303632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f31b2e3-cd0d-410a-a336-dfe77cdafc1a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:07.362967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffabf4-0418-43a7-bff1-2296106b475a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b48df06ca445caf9e447dbd79003c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4089410a0540402c816b8b884c2587f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:07.364118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffabf4-0418-43a7-bff1-2296106b475a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:07.364415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffabf4-0418-43a7-bff1-2296106b475a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:07.364697 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffabf4-0418-43a7-bff1-2296106b475a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:07.365424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffabf4-0418-43a7-bff1-2296106b475a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:25:07.365734 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffabf4-0418-43a7-bff1-2296106b475a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:07.367346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffabf4-0418-43a7-bff1-2296106b475a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:07.379632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1374/2733] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:25:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:25:07.420092 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3f6ed-e454-49da-be84-0956497a7b8e tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:07.431066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3f6ed-e454-49da-be84-0956497a7b8e tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:07.435475 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4b48df06ca445caf9e447dbd79003c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4089410a0540402c816b8b884c2587f1', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:07.436339 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:07.436627 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:07.436903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] PATH_INFO: `/v3/projects/7edc50527c094d5fac0387e46f734b40` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:07.438065 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7edc50527c094d5fac0387e46f734b40)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:07.440016 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:07.476722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:07.488489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7edc50527c094d5fac0387e46f734b40', 'request_id': 'req-69b041fb-8394-4915-adbb-6fb2890712ae'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:07.495692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7edc50527c094d5fac0387e46f734b40', 'request_id': 'req-69b041fb-8394-4915-adbb-6fb2890712ae'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:07.502256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7edc50527c094d5fac0387e46f734b40', 'request_id': 'req-69b041fb-8394-4915-adbb-6fb2890712ae'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:07.512209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] The token cache is being invalidate because project 7edc50527c094d5fac0387e46f734b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:07.512495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:07.513122 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-69b041fb-8394-4915-adbb-6fb2890712ae tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:07.514674 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1360/2734] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:07 2022] DELETE /identity/v3/projects/7edc50527c094d5fac0387e46f734b40 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:07.706981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffabf4-0418-43a7-bff1-2296106b475a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:07.708823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f1ffabf4-0418-43a7-bff1-2296106b475a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:07.774164 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffabf4-0418-43a7-bff1-2296106b475a tempest-VolumeQuotaClassesTest-208685522 tempest-VolumeQuotaClassesTest-208685522-project-admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:07.829753 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:07.831149 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:07.831435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:07.831712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] PATH_INFO: `/v3/users/9db349ff9e0b4a92b7363313e2eb8016` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:07.832916 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db349ff9e0b4a92b7363313e2eb8016)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:07.834839 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:07.870278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:07.885154 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db349ff9e0b4a92b7363313e2eb8016'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:07.885811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db349ff9e0b4a92b7363313e2eb8016'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:07.886398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db349ff9e0b4a92b7363313e2eb8016'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:07.894296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db349ff9e0b4a92b7363313e2eb8016'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:07.902961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db349ff9e0b4a92b7363313e2eb8016'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:07.921677 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1375/2735] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:07 2022] DELETE /identity/v3/users/9db349ff9e0b4a92b7363313e2eb8016 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:07.927007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287da8e-50e4-4ea1-b680-200f3deb3b77 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:07.927400 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287da8e-50e4-4ea1-b680-200f3deb3b77 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:07.927687 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287da8e-50e4-4ea1-b680-200f3deb3b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:07.945216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b287da8e-50e4-4ea1-b680-200f3deb3b77 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:07.946434 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b287da8e-50e4-4ea1-b680-200f3deb3b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:08.011655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1361/2736] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:08.022939 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:08.025150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:08.084124 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d679e48-e4f6-488b-b28d-f3c1bfc7e64e admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:08.087762 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ee51d-2bb5-4450-99c5-fc27e3af09f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:08.088406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ee51d-2bb5-4450-99c5-fc27e3af09f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:08.088690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ee51d-2bb5-4450-99c5-fc27e3af09f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:08.088966 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ee51d-2bb5-4450-99c5-fc27e3af09f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:08.098779 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ee51d-2bb5-4450-99c5-fc27e3af09f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:08.100256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ee51d-2bb5-4450-99c5-fc27e3af09f2 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:08.109828 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1376/2737] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:08.384442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287da8e-50e4-4ea1-b680-200f3deb3b77 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:08.386105 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b287da8e-50e4-4ea1-b680-200f3deb3b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:08.403946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b287da8e-50e4-4ea1-b680-200f3deb3b77 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:08.410189 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:08.411346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:08.412182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:08.412727 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] PATH_INFO: `/v3/projects/3a74db7002a74155b9d0a7f2791109fc` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:08.414246 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a74db7002a74155b9d0a7f2791109fc)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:08.416190 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:08.464659 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a74db7002a74155b9d0a7f2791109fc', 'request_id': 'req-14972e7d-a2a2-4068-834a-c449a038fb07'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:08.472746 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a74db7002a74155b9d0a7f2791109fc', 'request_id': 'req-14972e7d-a2a2-4068-834a-c449a038fb07'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:08.479539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a74db7002a74155b9d0a7f2791109fc', 'request_id': 'req-14972e7d-a2a2-4068-834a-c449a038fb07'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:08.490488 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] The token cache is being invalidate because project 3a74db7002a74155b9d0a7f2791109fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:08.490782 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:08.491513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:08.493040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1362/2738] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:08 2022] DELETE /identity/v3/projects/3a74db7002a74155b9d0a7f2791109fc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:08.497226 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ee51d-2bb5-4450-99c5-fc27e3af09f2 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:08.507485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ee51d-2bb5-4450-99c5-fc27e3af09f2 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:08.564413 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:08.565929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:08.566220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:08.566490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] PATH_INFO: `/v3/users/1f4b48df06ca445caf9e447dbd79003c` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:08.567688 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f4b48df06ca445caf9e447dbd79003c)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:08.569598 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:08.613788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f4b48df06ca445caf9e447dbd79003c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:08.614435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f4b48df06ca445caf9e447dbd79003c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:08.615035 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f4b48df06ca445caf9e447dbd79003c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:08.620424 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:08.622728 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f4b48df06ca445caf9e447dbd79003c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:08.632478 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8666f415-3c1c-44db-9349-f8f746eadf53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f4b48df06ca445caf9e447dbd79003c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:08.650194 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1377/2739] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:08 2022] DELETE /identity/v3/users/1f4b48df06ca445caf9e447dbd79003c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:08.913315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:08.915315 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:08.931153 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14972e7d-a2a2-4068-834a-c449a038fb07 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:08.977871 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:08.992490 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:08.993293 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:08.993576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:08.993852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] PATH_INFO: `/v3/projects/4089410a0540402c816b8b884c2587f1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:08.995017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4089410a0540402c816b8b884c2587f1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:08.996948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.041704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4089410a0540402c816b8b884c2587f1', 'request_id': 'req-2633571b-b2f5-40b9-ae0e-db08e633b186'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:09.048639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4089410a0540402c816b8b884c2587f1', 'request_id': 'req-2633571b-b2f5-40b9-ae0e-db08e633b186'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:09.055582 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4089410a0540402c816b8b884c2587f1', 'request_id': 'req-2633571b-b2f5-40b9-ae0e-db08e633b186'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:09.065906 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] The token cache is being invalidate because project 4089410a0540402c816b8b884c2587f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:09.066195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:09.066915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2633571b-b2f5-40b9-ae0e-db08e633b186 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:09.068495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1363/2740] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:08 2022] DELETE /identity/v3/projects/4089410a0540402c816b8b884c2587f1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:09.073430 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fab27-307c-425b-a784-3f1b0da7cc5a None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.073889 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fab27-307c-425b-a784-3f1b0da7cc5a None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.074023 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fab27-307c-425b-a784-3f1b0da7cc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.091300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-329fab27-307c-425b-a784-3f1b0da7cc5a None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:09.092776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-329fab27-307c-425b-a784-3f1b0da7cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:09.150212 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1378/2741] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:09.157765 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5829c9-57b7-4ea9-b0f8-ed7c897b1206 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.158055 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5829c9-57b7-4ea9-b0f8-ed7c897b1206 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.158337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5829c9-57b7-4ea9-b0f8-ed7c897b1206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.177130 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-9c5829c9-57b7-4ea9-b0f8-ed7c897b1206 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:09.178310 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-9c5829c9-57b7-4ea9-b0f8-ed7c897b1206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:09.186101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1364/2742] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:09.190170 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fab27-307c-425b-a784-3f1b0da7cc5a None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.200023 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fab27-307c-425b-a784-3f1b0da7cc5a None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.205128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67a58c-1813-4aaa-a0c3-f256575b7a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.205838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67a58c-1813-4aaa-a0c3-f256575b7a07 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.206128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67a58c-1813-4aaa-a0c3-f256575b7a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.206408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b67a58c-1813-4aaa-a0c3-f256575b7a07 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.207182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67a58c-1813-4aaa-a0c3-f256575b7a07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:25:09.207500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67a58c-1813-4aaa-a0c3-f256575b7a07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.209115 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b67a58c-1813-4aaa-a0c3-f256575b7a07 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.222151 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1379/2743] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:25:09.226633 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5829c9-57b7-4ea9-b0f8-ed7c897b1206 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.235738 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5829c9-57b7-4ea9-b0f8-ed7c897b1206 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.241382 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce04f-1be0-482d-855e-2b5eaaf58a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.242045 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce04f-1be0-482d-855e-2b5eaaf58a67 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.242328 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce04f-1be0-482d-855e-2b5eaaf58a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.242663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce04f-1be0-482d-855e-2b5eaaf58a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.243343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ce04f-1be0-482d-855e-2b5eaaf58a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.245247 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ce04f-1be0-482d-855e-2b5eaaf58a67 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.274255 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1365/2744] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:09 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:09.275545 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67a58c-1813-4aaa-a0c3-f256575b7a07 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.286206 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b67a58c-1813-4aaa-a0c3-f256575b7a07 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.342421 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b3c8f-46d2-489c-8de5-7e9d26a583d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.343117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b3c8f-46d2-489c-8de5-7e9d26a583d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.343503 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b3c8f-46d2-489c-8de5-7e9d26a583d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.343667 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b3c8f-46d2-489c-8de5-7e9d26a583d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.344381 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b3c8f-46d2-489c-8de5-7e9d26a583d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.346853 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b3c8f-46d2-489c-8de5-7e9d26a583d3 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.371084 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1380/2745] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:09 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:09.373291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce04f-1be0-482d-855e-2b5eaaf58a67 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.383899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce04f-1be0-482d-855e-2b5eaaf58a67 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.388514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb95dc4-53d6-4f2e-bf4f-422060aa756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.389175 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb95dc4-53d6-4f2e-bf4f-422060aa756a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.389457 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb95dc4-53d6-4f2e-bf4f-422060aa756a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.389734 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb95dc4-53d6-4f2e-bf4f-422060aa756a admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.390492 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb95dc4-53d6-4f2e-bf4f-422060aa756a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.392118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb95dc4-53d6-4f2e-bf4f-422060aa756a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.402963 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1366/2746] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:09.407133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b3c8f-46d2-489c-8de5-7e9d26a583d3 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.417931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b3c8f-46d2-489c-8de5-7e9d26a583d3 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.424335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ce6dc-7a26-4de6-9736-0737203580aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.424335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce6dc-7a26-4de6-9736-0737203580aa admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.424335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce6dc-7a26-4de6-9736-0737203580aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.424662 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ce6dc-7a26-4de6-9736-0737203580aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.425272 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ce6dc-7a26-4de6-9736-0737203580aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.426925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510ce6dc-7a26-4de6-9736-0737203580aa admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.440419 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1381/2747] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:09.441599 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb95dc4-53d6-4f2e-bf4f-422060aa756a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.452915 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb95dc4-53d6-4f2e-bf4f-422060aa756a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.458057 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96be0d-f979-4ceb-800a-9ce6f354e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.458953 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96be0d-f979-4ceb-800a-9ce6f354e46d admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.459290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96be0d-f979-4ceb-800a-9ce6f354e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.459574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e96be0d-f979-4ceb-800a-9ce6f354e46d admin admin] PATH_INFO: `/v3/projects/cb4e96f012ca44499cda392a481c954e/users/9426a09c9fe94b2a959845a36f7d122c/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.475683 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96be0d-f979-4ceb-800a-9ce6f354e46d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb4e96f012ca44499cda392a481c954e, user_id=9426a09c9fe94b2a959845a36f7d122c, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.478360 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e96be0d-f979-4ceb-800a-9ce6f354e46d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.491629 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1367/2748] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:09 2022] PUT /identity/v3/projects/cb4e96f012ca44499cda392a481c954e/users/9426a09c9fe94b2a959845a36f7d122c/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:25:09.503119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666e489-b99c-4904-b540-f6e1296c9045 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.503364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666e489-b99c-4904-b540-f6e1296c9045 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.503648 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666e489-b99c-4904-b540-f6e1296c9045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.529679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-6666e489-b99c-4904-b540-f6e1296c9045 None None] MFA Rules not processed for user `9426a09c9fe94b2a959845a36f7d122c`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:09.530872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6666e489-b99c-4904-b540-f6e1296c9045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:09.556335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:09.597220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1382/2749] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:09.601720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96be0d-f979-4ceb-800a-9ce6f354e46d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.664462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e96be0d-f979-4ceb-800a-9ce6f354e46d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.669613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc988d-8528-4de9-8e81-f58ca376eb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.670298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc988d-8528-4de9-8e81-f58ca376eb6e admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.670581 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc988d-8528-4de9-8e81-f58ca376eb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.670858 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc988d-8528-4de9-8e81-f58ca376eb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.671596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc988d-8528-4de9-8e81-f58ca376eb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.673496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc988d-8528-4de9-8e81-f58ca376eb6e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.693777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:09.704685 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1368/2750] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:09 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:09.706445 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666e489-b99c-4904-b540-f6e1296c9045 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.717496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666e489-b99c-4904-b540-f6e1296c9045 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.778106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09580f0-9c5c-4e12-8642-13506999fc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.778823 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09580f0-9c5c-4e12-8642-13506999fc96 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.779150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09580f0-9c5c-4e12-8642-13506999fc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.779442 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09580f0-9c5c-4e12-8642-13506999fc96 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.780497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09580f0-9c5c-4e12-8642-13506999fc96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.783261 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09580f0-9c5c-4e12-8642-13506999fc96 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.802207 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1383/2751] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:09 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:09.806251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc988d-8528-4de9-8e81-f58ca376eb6e admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.816967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc988d-8528-4de9-8e81-f58ca376eb6e admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.822104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04e473-52ef-4531-b3ec-6fa5a43a0115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.822766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e473-52ef-4531-b3ec-6fa5a43a0115 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.823075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e473-52ef-4531-b3ec-6fa5a43a0115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.823365 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04e473-52ef-4531-b3ec-6fa5a43a0115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.824097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04e473-52ef-4531-b3ec-6fa5a43a0115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.825698 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04e473-52ef-4531-b3ec-6fa5a43a0115 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.837235 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1369/2752] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:09.841263 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09580f0-9c5c-4e12-8642-13506999fc96 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.851376 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09580f0-9c5c-4e12-8642-13506999fc96 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.855978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f29969-3dd2-4254-942a-f27777b43bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.856842 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f29969-3dd2-4254-942a-f27777b43bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.857126 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f29969-3dd2-4254-942a-f27777b43bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.857401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f29969-3dd2-4254-942a-f27777b43bf7 admin admin] PATH_INFO: `/v3/projects/a984e194f61341bfb4d1ae3df960c87b/users/755b83fd70bb44eaa4f99f5428a912e2/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.871730 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f29969-3dd2-4254-942a-f27777b43bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a984e194f61341bfb4d1ae3df960c87b, user_id=755b83fd70bb44eaa4f99f5428a912e2, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.874387 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f29969-3dd2-4254-942a-f27777b43bf7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.887026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1384/2753] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:09 2022] PUT /identity/v3/projects/a984e194f61341bfb4d1ae3df960c87b/users/755b83fd70bb44eaa4f99f5428a912e2/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:09.890856 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04e473-52ef-4531-b3ec-6fa5a43a0115 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.901641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04e473-52ef-4531-b3ec-6fa5a43a0115 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.957676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59ecda-d6a3-49b5-ad23-d0e7fa34d834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.958398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59ecda-d6a3-49b5-ad23-d0e7fa34d834 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.958689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59ecda-d6a3-49b5-ad23-d0e7fa34d834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.958967 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59ecda-d6a3-49b5-ad23-d0e7fa34d834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.959717 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59ecda-d6a3-49b5-ad23-d0e7fa34d834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.961308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59ecda-d6a3-49b5-ad23-d0e7fa34d834 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:09.971331 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1370/2754] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:09.976167 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f29969-3dd2-4254-942a-f27777b43bf7 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:09.986230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f29969-3dd2-4254-942a-f27777b43bf7 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:09.990898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe5fc0-cb62-4c74-b614-93db82872d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:09.991805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe5fc0-cb62-4c74-b614-93db82872d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:09.992088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe5fc0-cb62-4c74-b614-93db82872d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:09.992364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe5fc0-cb62-4c74-b614-93db82872d96 admin admin] PATH_INFO: `/v3/projects/a984e194f61341bfb4d1ae3df960c87b/users/755b83fd70bb44eaa4f99f5428a912e2/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:09.994331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe5fc0-cb62-4c74-b614-93db82872d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a984e194f61341bfb4d1ae3df960c87b, user_id=755b83fd70bb44eaa4f99f5428a912e2, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:09.996988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe5fc0-cb62-4c74-b614-93db82872d96 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:10.007530 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1385/2755] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:09 2022] PUT /identity/v3/projects/a984e194f61341bfb4d1ae3df960c87b/users/755b83fd70bb44eaa4f99f5428a912e2/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:10.017837 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21b107-00be-4a21-9fee-5bb8275b68b2 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:10.018773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21b107-00be-4a21-9fee-5bb8275b68b2 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:10.019086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f21b107-00be-4a21-9fee-5bb8275b68b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:10.044917 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0f21b107-00be-4a21-9fee-5bb8275b68b2 None None] MFA Rules not processed for user `755b83fd70bb44eaa4f99f5428a912e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:10.046054 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0f21b107-00be-4a21-9fee-5bb8275b68b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:10.114973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1371/2756] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:10 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:10.125776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe5fc0-cb62-4c74-b614-93db82872d96 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:10.128209 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-21fe5fc0-cb62-4c74-b614-93db82872d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:10.183086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe5fc0-cb62-4c74-b614-93db82872d96 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:10.186700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed673ea-63b6-4a34-92f9-62019235e46d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:10.187390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed673ea-63b6-4a34-92f9-62019235e46d service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:10.187678 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed673ea-63b6-4a34-92f9-62019235e46d service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:10.187961 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed673ea-63b6-4a34-92f9-62019235e46d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:10.197810 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed673ea-63b6-4a34-92f9-62019235e46d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:10.199339 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed673ea-63b6-4a34-92f9-62019235e46d service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:10.209083 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1386/2757] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:10 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:10.255896 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21b107-00be-4a21-9fee-5bb8275b68b2 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:10.257323 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0f21b107-00be-4a21-9fee-5bb8275b68b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:10.269171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f21b107-00be-4a21-9fee-5bb8275b68b2 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:10.272803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea970883-7531-4b64-815c-3eb3ff124b09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:10.273442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea970883-7531-4b64-815c-3eb3ff124b09 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:10.273732 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea970883-7531-4b64-815c-3eb3ff124b09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:10.274014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea970883-7531-4b64-815c-3eb3ff124b09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:10.329941 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea970883-7531-4b64-815c-3eb3ff124b09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:10.331394 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea970883-7531-4b64-815c-3eb3ff124b09 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:10.340800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1372/2758] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:10.666617 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed673ea-63b6-4a34-92f9-62019235e46d service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:10.668097 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8ed673ea-63b6-4a34-92f9-62019235e46d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:10.695702 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:10.734679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed673ea-63b6-4a34-92f9-62019235e46d service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:10.739391 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:10.740258 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:10.740547 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:10.740831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] PATH_INFO: `/v3/users/a8f124e18c644815bff0d95e52ad667f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:10.742013 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f124e18c644815bff0d95e52ad667f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:10.744015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:10.789289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f124e18c644815bff0d95e52ad667f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:10.789953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f124e18c644815bff0d95e52ad667f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:10.790552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f124e18c644815bff0d95e52ad667f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:10.797327 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f124e18c644815bff0d95e52ad667f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:10.804143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f124e18c644815bff0d95e52ad667f'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:10.821681 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1387/2759] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:10 2022] DELETE /identity/v3/users/a8f124e18c644815bff0d95e52ad667f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:10.826207 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afff861-0452-4e31-853b-8a1c2b042ade None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:10.826672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afff861-0452-4e31-853b-8a1c2b042ade None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:10.826785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afff861-0452-4e31-853b-8a1c2b042ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:10.838503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:10.844668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-8afff861-0452-4e31-853b-8a1c2b042ade None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:10.845864 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8afff861-0452-4e31-853b-8a1c2b042ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:10.906556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1373/2760] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:10.915999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:10.917405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:10.974865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46ccf9-b770-4e4f-a148-b30ae5c95da8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:10.978488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52092096-170e-4131-afe7-bbe59b50da9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:10.979160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52092096-170e-4131-afe7-bbe59b50da9f service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:10.979447 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52092096-170e-4131-afe7-bbe59b50da9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:10.979725 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52092096-170e-4131-afe7-bbe59b50da9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:10.989074 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52092096-170e-4131-afe7-bbe59b50da9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:10.990501 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52092096-170e-4131-afe7-bbe59b50da9f service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:11.000566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1388/2761] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:11.262477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afff861-0452-4e31-853b-8a1c2b042ade None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:11.263940 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-8afff861-0452-4e31-853b-8a1c2b042ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:11.277537 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afff861-0452-4e31-853b-8a1c2b042ade None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:11.282584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:11.284135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:11.284441 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:11.284676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] PATH_INFO: `/v3/projects/387e3e504cdb4abba8aaae2ab32a8ddb` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:11.285973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387e3e504cdb4abba8aaae2ab32a8ddb)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:11.287797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:11.333331 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387e3e504cdb4abba8aaae2ab32a8ddb', 'request_id': 'req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.340159 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387e3e504cdb4abba8aaae2ab32a8ddb', 'request_id': 'req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.347361 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387e3e504cdb4abba8aaae2ab32a8ddb', 'request_id': 'req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.357462 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] The token cache is being invalidate because project 387e3e504cdb4abba8aaae2ab32a8ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:11.357752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.358378 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.359928 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1374/2762] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:11 2022] DELETE /identity/v3/projects/387e3e504cdb4abba8aaae2ab32a8ddb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:11.364504 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52092096-170e-4131-afe7-bbe59b50da9f service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:11.374178 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52092096-170e-4131-afe7-bbe59b50da9f service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:11.415329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:11.432423 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:11.433292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:11.433580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:11.433864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] PATH_INFO: `/v3/users/604d74a5fbc241f49cb67aa1628d6bd0` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:11.435073 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=604d74a5fbc241f49cb67aa1628d6bd0)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:11.436999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:11.481198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '604d74a5fbc241f49cb67aa1628d6bd0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.481831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '604d74a5fbc241f49cb67aa1628d6bd0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.482422 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '604d74a5fbc241f49cb67aa1628d6bd0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.490551 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '604d74a5fbc241f49cb67aa1628d6bd0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.497296 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ab9487ab-914b-44a9-bb13-95b303eed1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '604d74a5fbc241f49cb67aa1628d6bd0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.514055 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1389/2763] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:11 2022] DELETE /identity/v3/users/604d74a5fbc241f49cb67aa1628d6bd0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:11.791412 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:11.792948 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:11.809622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55259cfd-fb20-44e1-b5c4-35bb7de7efa1 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:11.870107 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:11.870897 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:11.871215 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:11.871496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] PATH_INFO: `/v3/projects/e7638329af1e44f5bfea88816c039267` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:11.872692 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7638329af1e44f5bfea88816c039267)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:11.874617 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:11.909740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:11.923811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7638329af1e44f5bfea88816c039267', 'request_id': 'req-8d3ae49e-ff75-4e80-b9ad-163670b9774d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.935066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7638329af1e44f5bfea88816c039267', 'request_id': 'req-8d3ae49e-ff75-4e80-b9ad-163670b9774d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.942523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7638329af1e44f5bfea88816c039267', 'request_id': 'req-8d3ae49e-ff75-4e80-b9ad-163670b9774d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.957551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] The token cache is being invalidate because project e7638329af1e44f5bfea88816c039267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:11.957848 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.958523 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-8d3ae49e-ff75-4e80-b9ad-163670b9774d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:11.960081 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1375/2764] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:11 2022] DELETE /identity/v3/projects/e7638329af1e44f5bfea88816c039267 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:11.964840 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e6392-fc0c-45f9-b92c-3d091587506c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:11.965131 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e6392-fc0c-45f9-b92c-3d091587506c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:11.965437 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e6392-fc0c-45f9-b92c-3d091587506c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:11.984479 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-5d9e6392-fc0c-45f9-b92c-3d091587506c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:11.985683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5d9e6392-fc0c-45f9-b92c-3d091587506c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:12.051367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1390/2765] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:12.059354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954456e-42f0-467c-b777-e7d6ddd5b602 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.059551 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954456e-42f0-467c-b777-e7d6ddd5b602 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.059847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2954456e-42f0-467c-b777-e7d6ddd5b602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.078276 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-2954456e-42f0-467c-b777-e7d6ddd5b602 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:12.079507 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2954456e-42f0-467c-b777-e7d6ddd5b602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:12.089789 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1376/2766] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:25:12.094628 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e6392-fc0c-45f9-b92c-3d091587506c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.104977 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e6392-fc0c-45f9-b92c-3d091587506c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.110022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05501689-5fba-426c-8340-0cabf11fc1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.110699 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501689-5fba-426c-8340-0cabf11fc1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.111370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501689-5fba-426c-8340-0cabf11fc1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.111370 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05501689-5fba-426c-8340-0cabf11fc1d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.112067 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05501689-5fba-426c-8340-0cabf11fc1d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:25:12.112378 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05501689-5fba-426c-8340-0cabf11fc1d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.113986 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05501689-5fba-426c-8340-0cabf11fc1d7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.126306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1391/2767] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:25:12.131174 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954456e-42f0-467c-b777-e7d6ddd5b602 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.140297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2954456e-42f0-467c-b777-e7d6ddd5b602 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.145658 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757b2af-c493-4f60-9688-5a5c35c8ae1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.146314 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757b2af-c493-4f60-9688-5a5c35c8ae1b admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.146602 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757b2af-c493-4f60-9688-5a5c35c8ae1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.146884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757b2af-c493-4f60-9688-5a5c35c8ae1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.147622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757b2af-c493-4f60-9688-5a5c35c8ae1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.149545 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757b2af-c493-4f60-9688-5a5c35c8ae1b admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.178950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1377/2768] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:12 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:12.180111 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05501689-5fba-426c-8340-0cabf11fc1d7 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.191272 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05501689-5fba-426c-8340-0cabf11fc1d7 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.244747 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:12.257058 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0343dc-944a-4934-b3f1-e20b2a2e7b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.257819 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0343dc-944a-4934-b3f1-e20b2a2e7b0c admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.258410 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0343dc-944a-4934-b3f1-e20b2a2e7b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.258825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0343dc-944a-4934-b3f1-e20b2a2e7b0c admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.259582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0343dc-944a-4934-b3f1-e20b2a2e7b0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.261500 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0343dc-944a-4934-b3f1-e20b2a2e7b0c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.283189 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1392/2769] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:12 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:12.287380 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757b2af-c493-4f60-9688-5a5c35c8ae1b admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.297887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757b2af-c493-4f60-9688-5a5c35c8ae1b admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.303166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15fef6-8a27-49cb-99e8-e43754eb06d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.303849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15fef6-8a27-49cb-99e8-e43754eb06d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.304142 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15fef6-8a27-49cb-99e8-e43754eb06d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.304416 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15fef6-8a27-49cb-99e8-e43754eb06d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.305131 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15fef6-8a27-49cb-99e8-e43754eb06d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.306722 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15fef6-8a27-49cb-99e8-e43754eb06d5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.317610 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1378/2770] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:25:12.322148 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0343dc-944a-4934-b3f1-e20b2a2e7b0c admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.336357 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0343dc-944a-4934-b3f1-e20b2a2e7b0c admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.342610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720d158-c98a-46cb-98f1-d20cd3562f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.343346 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720d158-c98a-46cb-98f1-d20cd3562f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.343652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720d158-c98a-46cb-98f1-d20cd3562f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.343935 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720d158-c98a-46cb-98f1-d20cd3562f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.344693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720d158-c98a-46cb-98f1-d20cd3562f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.346324 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720d158-c98a-46cb-98f1-d20cd3562f45 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.356485 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1393/2771] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:12.360630 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15fef6-8a27-49cb-99e8-e43754eb06d5 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.374444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15fef6-8a27-49cb-99e8-e43754eb06d5 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.380319 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d409041-d025-43c4-adba-38ee42985f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.381279 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d409041-d025-43c4-adba-38ee42985f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.381632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d409041-d025-43c4-adba-38ee42985f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.381913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d409041-d025-43c4-adba-38ee42985f97 admin admin] PATH_INFO: `/v3/projects/9cd66a57f7064f3b8026a5c979f9fbcf/users/1e1159f305354edeab692e8f2a1fc87d/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.399591 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d409041-d025-43c4-adba-38ee42985f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cd66a57f7064f3b8026a5c979f9fbcf, user_id=1e1159f305354edeab692e8f2a1fc87d, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.402317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d409041-d025-43c4-adba-38ee42985f97 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.416496 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1379/2772] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:12 2022] PUT /identity/v3/projects/9cd66a57f7064f3b8026a5c979f9fbcf/users/1e1159f305354edeab692e8f2a1fc87d/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:12.428262 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb6149-470e-459b-8958-c6ec12d5af06 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.428745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb6149-470e-459b-8958-c6ec12d5af06 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.428833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb6149-470e-459b-8958-c6ec12d5af06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.455896 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-efcb6149-470e-459b-8958-c6ec12d5af06 None None] MFA Rules not processed for user `1e1159f305354edeab692e8f2a1fc87d`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:12.457166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-efcb6149-470e-459b-8958-c6ec12d5af06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:12.519930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1394/2773] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:12.524307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d409041-d025-43c4-adba-38ee42985f97 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.583307 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d409041-d025-43c4-adba-38ee42985f97 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.587862 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0325f-e64f-4929-b5dc-a9e89bea8b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.588514 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0325f-e64f-4929-b5dc-a9e89bea8b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.588796 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0325f-e64f-4929-b5dc-a9e89bea8b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.589072 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0325f-e64f-4929-b5dc-a9e89bea8b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.589755 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0325f-e64f-4929-b5dc-a9e89bea8b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.591704 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0325f-e64f-4929-b5dc-a9e89bea8b53 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.627285 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1380/2774] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:12 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:12.631849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb6149-470e-459b-8958-c6ec12d5af06 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.642987 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb6149-470e-459b-8958-c6ec12d5af06 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.701564 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660468dc-947d-4ec0-8168-2c45030d153a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.702247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660468dc-947d-4ec0-8168-2c45030d153a admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.702542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660468dc-947d-4ec0-8168-2c45030d153a admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.702844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660468dc-947d-4ec0-8168-2c45030d153a admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.703636 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660468dc-947d-4ec0-8168-2c45030d153a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.705566 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660468dc-947d-4ec0-8168-2c45030d153a admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.730264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1395/2775] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:12 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:12.731571 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0325f-e64f-4929-b5dc-a9e89bea8b53 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.743878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0325f-e64f-4929-b5dc-a9e89bea8b53 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.748628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ede2c3-4d9c-4a53-9263-a21b6eb09fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.749291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ede2c3-4d9c-4a53-9263-a21b6eb09fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.749575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ede2c3-4d9c-4a53-9263-a21b6eb09fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.749857 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ede2c3-4d9c-4a53-9263-a21b6eb09fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.750616 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ede2c3-4d9c-4a53-9263-a21b6eb09fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.752316 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ede2c3-4d9c-4a53-9263-a21b6eb09fe3 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.763367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1381/2776] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:12.767715 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660468dc-947d-4ec0-8168-2c45030d153a admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.778076 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660468dc-947d-4ec0-8168-2c45030d153a admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.782685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97ba28-c4e7-47d5-b97d-069e12a14438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.784099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97ba28-c4e7-47d5-b97d-069e12a14438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.784382 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97ba28-c4e7-47d5-b97d-069e12a14438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.784659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97ba28-c4e7-47d5-b97d-069e12a14438 admin admin] PATH_INFO: `/v3/projects/768759bc129b49b48865d2ff295c86ff/users/6a08ca4e147d4221acbb4d7c52979b74/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.799771 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97ba28-c4e7-47d5-b97d-069e12a14438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=768759bc129b49b48865d2ff295c86ff, user_id=6a08ca4e147d4221acbb4d7c52979b74, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.802435 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97ba28-c4e7-47d5-b97d-069e12a14438 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.815468 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1396/2777] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:12 2022] PUT /identity/v3/projects/768759bc129b49b48865d2ff295c86ff/users/6a08ca4e147d4221acbb4d7c52979b74/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:12.819593 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ede2c3-4d9c-4a53-9263-a21b6eb09fe3 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.829833 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ede2c3-4d9c-4a53-9263-a21b6eb09fe3 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.870733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:12.899718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059d010-482d-4034-9499-032a9dc86ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.900388 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059d010-482d-4034-9499-032a9dc86ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.900744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059d010-482d-4034-9499-032a9dc86ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.901093 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059d010-482d-4034-9499-032a9dc86ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.901820 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059d010-482d-4034-9499-032a9dc86ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.903474 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059d010-482d-4034-9499-032a9dc86ee0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.913695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1382/2778] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:12.918082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97ba28-c4e7-47d5-b97d-069e12a14438 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:12.929002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97ba28-c4e7-47d5-b97d-069e12a14438 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:12.933712 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f467c1-5d71-4436-92b9-496aa982e288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:12.934590 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f467c1-5d71-4436-92b9-496aa982e288 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.934877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f467c1-5d71-4436-92b9-496aa982e288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.935201 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f467c1-5d71-4436-92b9-496aa982e288 admin admin] PATH_INFO: `/v3/projects/768759bc129b49b48865d2ff295c86ff/users/6a08ca4e147d4221acbb4d7c52979b74/roles/d1d68fb96ed04adf9cb22d55ba78f92f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.937213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f467c1-5d71-4436-92b9-496aa982e288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=768759bc129b49b48865d2ff295c86ff, user_id=6a08ca4e147d4221acbb4d7c52979b74, role_id=d1d68fb96ed04adf9cb22d55ba78f92f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:12.940025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f467c1-5d71-4436-92b9-496aa982e288 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:12.950955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1397/2779] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:12 2022] PUT /identity/v3/projects/768759bc129b49b48865d2ff295c86ff/users/6a08ca4e147d4221acbb4d7c52979b74/roles/d1d68fb96ed04adf9cb22d55ba78f92f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:25:12.961413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a4315-fd19-4c93-a8b0-709748fd556e None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:12.961707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a4315-fd19-4c93-a8b0-709748fd556e None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:12.961985 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a4315-fd19-4c93-a8b0-709748fd556e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:12.990117 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-097a4315-fd19-4c93-a8b0-709748fd556e None None] MFA Rules not processed for user `6a08ca4e147d4221acbb4d7c52979b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:12.991559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-097a4315-fd19-4c93-a8b0-709748fd556e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:13.078498 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1383/2780] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:12 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:13.090277 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f467c1-5d71-4436-92b9-496aa982e288 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:13.092458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d9f467c1-5d71-4436-92b9-496aa982e288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:13.109116 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:13.165553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f467c1-5d71-4436-92b9-496aa982e288 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:13.177161 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db149dbf-ef61-46eb-a078-234d814cdfc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:13.178472 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db149dbf-ef61-46eb-a078-234d814cdfc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:13.178786 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db149dbf-ef61-46eb-a078-234d814cdfc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:13.179109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db149dbf-ef61-46eb-a078-234d814cdfc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:13.189048 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db149dbf-ef61-46eb-a078-234d814cdfc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:13.190517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db149dbf-ef61-46eb-a078-234d814cdfc9 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:13.201000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1398/2781] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:13.334337 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a4315-fd19-4c93-a8b0-709748fd556e None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:13.395910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a4315-fd19-4c93-a8b0-709748fd556e None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:13.400399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:13.401308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:13.401558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:13.401839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] PATH_INFO: `/v3/users/1e1159f305354edeab692e8f2a1fc87d` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:13.403014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e1159f305354edeab692e8f2a1fc87d)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:13.404976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:13.455618 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e1159f305354edeab692e8f2a1fc87d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:13.456597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e1159f305354edeab692e8f2a1fc87d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:13.457358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e1159f305354edeab692e8f2a1fc87d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:13.465322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e1159f305354edeab692e8f2a1fc87d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:13.466536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:13.476177 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e1159f305354edeab692e8f2a1fc87d'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:13.494565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1384/2782] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:13 2022] DELETE /identity/v3/users/1e1159f305354edeab692e8f2a1fc87d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:13.499018 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fc0be-9607-4e87-be8f-64517cebd260 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:13.499554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fc0be-9607-4e87-be8f-64517cebd260 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:13.499672 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fc0be-9607-4e87-be8f-64517cebd260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:13.516967 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-c97fc0be-9607-4e87-be8f-64517cebd260 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:13.518159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c97fc0be-9607-4e87-be8f-64517cebd260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:13.581723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1399/2783] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:13.591317 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:13.593920 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:13.606553 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fc0be-9607-4e87-be8f-64517cebd260 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:13.607995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c97fc0be-9607-4e87-be8f-64517cebd260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:13.620554 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fc0be-9607-4e87-be8f-64517cebd260 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:13.625132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:13.625990 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:13.626276 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:13.626556 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] PATH_INFO: `/v3/users/9426a09c9fe94b2a959845a36f7d122c` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:13.627806 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9426a09c9fe94b2a959845a36f7d122c)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:13.629727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:13.656181 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb32f88-a0f1-4b95-8740-6b18cb868926 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:13.660626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8ed49-52aa-4a5b-b076-11d781a3cfae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:13.661477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8ed49-52aa-4a5b-b076-11d781a3cfae service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:13.661819 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8ed49-52aa-4a5b-b076-11d781a3cfae service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:13.662169 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8ed49-52aa-4a5b-b076-11d781a3cfae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:13.672120 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8ed49-52aa-4a5b-b076-11d781a3cfae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:13.673576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8ed49-52aa-4a5b-b076-11d781a3cfae service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:13.677248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9426a09c9fe94b2a959845a36f7d122c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:13.677908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9426a09c9fe94b2a959845a36f7d122c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:13.678532 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9426a09c9fe94b2a959845a36f7d122c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:13.684216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1385/2784] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:13.685722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9426a09c9fe94b2a959845a36f7d122c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:13.693382 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9426a09c9fe94b2a959845a36f7d122c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:13.711412 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1400/2785] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:13 2022] DELETE /identity/v3/users/9426a09c9fe94b2a959845a36f7d122c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:25:13.716364 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97c06-c39e-493c-8700-20590e6b7281 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:13.716652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97c06-c39e-493c-8700-20590e6b7281 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:13.717182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b97c06-c39e-493c-8700-20590e6b7281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:13.735398 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-a2b97c06-c39e-493c-8700-20590e6b7281 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:13.737209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a2b97c06-c39e-493c-8700-20590e6b7281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:13.797739 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1386/2786] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:13.807264 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:13.808671 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:13.842453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:13.871101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e74ca-9e1a-4dc9-b1b6-72b54fcf02e7 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:13.874742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d5f2c-5bdb-458c-8de3-046da7c0fbac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:13.875456 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d5f2c-5bdb-458c-8de3-046da7c0fbac service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:13.875708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d5f2c-5bdb-458c-8de3-046da7c0fbac service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:13.875984 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33d5f2c-5bdb-458c-8de3-046da7c0fbac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:13.885060 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d5f2c-5bdb-458c-8de3-046da7c0fbac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:13.886512 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33d5f2c-5bdb-458c-8de3-046da7c0fbac service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:13.896415 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1401/2787] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:13.996185 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b97c06-c39e-493c-8700-20590e6b7281 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:13.998104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-a2b97c06-c39e-493c-8700-20590e6b7281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:14.011407 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b97c06-c39e-493c-8700-20590e6b7281 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:14.016935 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:14.018527 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:14.018672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:14.018838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] PATH_INFO: `/v3/projects/9cd66a57f7064f3b8026a5c979f9fbcf` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:14.020242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cd66a57f7064f3b8026a5c979f9fbcf)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:14.022162 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:14.074905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cd66a57f7064f3b8026a5c979f9fbcf', 'request_id': 'req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.083306 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cd66a57f7064f3b8026a5c979f9fbcf', 'request_id': 'req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.091669 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cd66a57f7064f3b8026a5c979f9fbcf', 'request_id': 'req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.099145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:14.104921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] The token cache is being invalidate because project 9cd66a57f7064f3b8026a5c979f9fbcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:14.105405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.106230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.109343 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1387/2788] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:13 2022] DELETE /identity/v3/projects/9cd66a57f7064f3b8026a5c979f9fbcf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:14.112377 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d5f2c-5bdb-458c-8de3-046da7c0fbac service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:14.122331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33d5f2c-5bdb-458c-8de3-046da7c0fbac service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:14.180384 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:14.181182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:14.181473 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:14.181756 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] PATH_INFO: `/v3/users/6a08ca4e147d4221acbb4d7c52979b74` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:14.183009 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a08ca4e147d4221acbb4d7c52979b74)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:14.185015 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:14.224831 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:14.226536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:14.246245 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a08ca4e147d4221acbb4d7c52979b74'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.247020 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a08ca4e147d4221acbb4d7c52979b74'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.247758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a08ca4e147d4221acbb4d7c52979b74'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.249168 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bd4b0-ec3d-4e1e-903e-f9758da34bd4 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:14.255233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:14.256103 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:14.256395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:14.256681 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] PATH_INFO: `/v3/projects/cb4e96f012ca44499cda392a481c954e` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:14.256907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a08ca4e147d4221acbb4d7c52979b74'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.257877 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb4e96f012ca44499cda392a481c954e)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:14.259836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:14.264844 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a08ca4e147d4221acbb4d7c52979b74'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.284751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1402/2789] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:14 2022] DELETE /identity/v3/users/6a08ca4e147d4221acbb4d7c52979b74 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:14.311636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb4e96f012ca44499cda392a481c954e', 'request_id': 'req-40fe4331-b904-4ada-811b-e7cdc06a6e99'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.319588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb4e96f012ca44499cda392a481c954e', 'request_id': 'req-40fe4331-b904-4ada-811b-e7cdc06a6e99'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.327080 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb4e96f012ca44499cda392a481c954e', 'request_id': 'req-40fe4331-b904-4ada-811b-e7cdc06a6e99'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.340532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] The token cache is being invalidate because project cb4e96f012ca44499cda392a481c954e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:14.341098 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.341883 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.344192 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1388/2790] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:14 2022] DELETE /identity/v3/projects/cb4e96f012ca44499cda392a481c954e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:14.348138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:14.349776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:14.422054 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:14.444238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d830880-2608-4a52-9532-8dfb05311365 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:14.502888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:14.503985 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:14.504369 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:14.504754 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] PATH_INFO: `/v3/users/755b83fd70bb44eaa4f99f5428a912e2` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:14.506070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755b83fd70bb44eaa4f99f5428a912e2)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:14.508160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:14.564685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755b83fd70bb44eaa4f99f5428a912e2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.565978 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755b83fd70bb44eaa4f99f5428a912e2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.566599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755b83fd70bb44eaa4f99f5428a912e2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.575446 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755b83fd70bb44eaa4f99f5428a912e2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.584928 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755b83fd70bb44eaa4f99f5428a912e2'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:14.604139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1403/2791] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:14 2022] DELETE /identity/v3/users/755b83fd70bb44eaa4f99f5428a912e2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:14.864827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:14.866358 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:14.886075 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe4331-b904-4ada-811b-e7cdc06a6e99 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:14.897691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:14.899289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:14.950596 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:14.951614 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:14.951973 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:14.952263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] PATH_INFO: `/v3/projects/768759bc129b49b48865d2ff295c86ff` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:14.953452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=768759bc129b49b48865d2ff295c86ff)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:14.955557 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:14.968731 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bf7fb-3c9d-4d9c-bc2a-be150c88eeb8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:14.974017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:14.974868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:14.975182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:14.975465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] PATH_INFO: `/v3/projects/a984e194f61341bfb4d1ae3df960c87b` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:14.976639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a984e194f61341bfb4d1ae3df960c87b)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:14.978663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:14.996817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:15.002826 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '768759bc129b49b48865d2ff295c86ff', 'request_id': 'req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.010950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '768759bc129b49b48865d2ff295c86ff', 'request_id': 'req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.017164 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '768759bc129b49b48865d2ff295c86ff', 'request_id': 'req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.028576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] The token cache is being invalidate because project 768759bc129b49b48865d2ff295c86ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:15.028873 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.029521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-888c9b08-cfa1-4da7-8ca2-c14e43af11c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.030104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a984e194f61341bfb4d1ae3df960c87b', 'request_id': 'req-a58a577b-b056-44b2-8bc0-0c3c471e8115'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.031161 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1389/2792] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:14 2022] DELETE /identity/v3/projects/768759bc129b49b48865d2ff295c86ff => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:15.036486 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f7180-27a8-431b-84b2-d50104e6194d None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.036878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f7180-27a8-431b-84b2-d50104e6194d None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.037166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f7180-27a8-431b-84b2-d50104e6194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.041622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a984e194f61341bfb4d1ae3df960c87b', 'request_id': 'req-a58a577b-b056-44b2-8bc0-0c3c471e8115'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.048307 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a984e194f61341bfb4d1ae3df960c87b', 'request_id': 'req-a58a577b-b056-44b2-8bc0-0c3c471e8115'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.057201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-150f7180-27a8-431b-84b2-d50104e6194d None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:15.058452 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-150f7180-27a8-431b-84b2-d50104e6194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:15.061022 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] The token cache is being invalidate because project a984e194f61341bfb4d1ae3df960c87b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:15.061426 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.062088 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-a58a577b-b056-44b2-8bc0-0c3c471e8115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.063703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1404/2793] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:14 2022] DELETE /identity/v3/projects/a984e194f61341bfb4d1ae3df960c87b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:15.064108 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:15.068774 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba546499-9cf2-4089-8287-594cc2ec7b85 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.069260 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba546499-9cf2-4089-8287-594cc2ec7b85 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.071394 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba546499-9cf2-4089-8287-594cc2ec7b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.098487 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ba546499-9cf2-4089-8287-594cc2ec7b85 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:15.099733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ba546499-9cf2-4089-8287-594cc2ec7b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:15.130718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1390/2794] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:25:15.138405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9e3c3-d06c-487b-9621-d7b6a621ba23 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.138697 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9e3c3-d06c-487b-9621-d7b6a621ba23 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.138979 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c9e3c3-d06c-487b-9621-d7b6a621ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.156209 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-27c9e3c3-d06c-487b-9621-d7b6a621ba23 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:15.157461 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-27c9e3c3-d06c-487b-9621-d7b6a621ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:15.160405 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1405/2795] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:15.165874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1391/2796] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:25:15.168019 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d236a6-1122-49cf-a73c-a5223b40489b None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.168311 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d236a6-1122-49cf-a73c-a5223b40489b None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.168594 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d236a6-1122-49cf-a73c-a5223b40489b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.169758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9e3c3-d06c-487b-9621-d7b6a621ba23 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.179740 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c9e3c3-d06c-487b-9621-d7b6a621ba23 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.184414 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daf626-a264-4dc5-b60b-b21fc23753b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.185071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daf626-a264-4dc5-b60b-b21fc23753b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.185359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daf626-a264-4dc5-b60b-b21fc23753b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.185645 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57daf626-a264-4dc5-b60b-b21fc23753b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.186549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57daf626-a264-4dc5-b60b-b21fc23753b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:25:15.186863 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57daf626-a264-4dc5-b60b-b21fc23753b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.188281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-52d236a6-1122-49cf-a73c-a5223b40489b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:15.188512 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57daf626-a264-4dc5-b60b-b21fc23753b2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.189711 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-52d236a6-1122-49cf-a73c-a5223b40489b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:15.201173 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1406/2797] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:15.201950 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1392/2798] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:25:15.202592 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d236a6-1122-49cf-a73c-a5223b40489b None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.205510 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daf626-a264-4dc5-b60b-b21fc23753b2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.212540 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d236a6-1122-49cf-a73c-a5223b40489b None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.215901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57daf626-a264-4dc5-b60b-b21fc23753b2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.217703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca978d8-0fb4-498c-aaff-ededc27ab886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.218372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca978d8-0fb4-498c-aaff-ededc27ab886 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.218659 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca978d8-0fb4-498c-aaff-ededc27ab886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.218992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca978d8-0fb4-498c-aaff-ededc27ab886 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.219698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca978d8-0fb4-498c-aaff-ededc27ab886 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:25:15.220008 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca978d8-0fb4-498c-aaff-ededc27ab886 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.221908 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e4478-f631-4a40-8aeb-18f66c0e498d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.222169 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca978d8-0fb4-498c-aaff-ededc27ab886 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.222565 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e4478-f631-4a40-8aeb-18f66c0e498d admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.222855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e4478-f631-4a40-8aeb-18f66c0e498d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.223171 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e4478-f631-4a40-8aeb-18f66c0e498d admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.223875 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e4478-f631-4a40-8aeb-18f66c0e498d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.225769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e4478-f631-4a40-8aeb-18f66c0e498d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.233827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1407/2799] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 05 15:25:15.238847 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca978d8-0fb4-498c-aaff-ededc27ab886 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.248684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca978d8-0fb4-498c-aaff-ededc27ab886 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.253274 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ee823-9a55-4226-b934-a2c38c8910de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.256420 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1393/2800] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:25:15.256562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ee823-9a55-4226-b934-a2c38c8910de admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.256562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ee823-9a55-4226-b934-a2c38c8910de admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.256562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7ee823-9a55-4226-b934-a2c38c8910de admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.256562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ee823-9a55-4226-b934-a2c38c8910de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.257313 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7ee823-9a55-4226-b934-a2c38c8910de admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.257424 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e4478-f631-4a40-8aeb-18f66c0e498d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.284790 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1408/2801] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:25:15.289180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ee823-9a55-4226-b934-a2c38c8910de admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.320673 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e4478-f631-4a40-8aeb-18f66c0e498d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.351068 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7ee823-9a55-4226-b934-a2c38c8910de admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.355769 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e4c7a-f0fd-4a64-ac41-5a58104bf52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.356427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e4c7a-f0fd-4a64-ac41-5a58104bf52d admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.356716 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e4c7a-f0fd-4a64-ac41-5a58104bf52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.356995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e4c7a-f0fd-4a64-ac41-5a58104bf52d admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.357685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e4c7a-f0fd-4a64-ac41-5a58104bf52d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.359638 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e4c7a-f0fd-4a64-ac41-5a58104bf52d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.368175 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:15.377444 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4572e7-25d6-411a-a420-2b964c4f52d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.378324 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4572e7-25d6-411a-a420-2b964c4f52d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.378615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4572e7-25d6-411a-a420-2b964c4f52d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.378893 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4572e7-25d6-411a-a420-2b964c4f52d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.379654 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4572e7-25d6-411a-a420-2b964c4f52d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.381598 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4572e7-25d6-411a-a420-2b964c4f52d5 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.382402 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1409/2802] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:25:15.386427 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e4c7a-f0fd-4a64-ac41-5a58104bf52d admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.398654 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e4c7a-f0fd-4a64-ac41-5a58104bf52d admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.402563 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1394/2803] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:15.403586 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794de7f-f837-437c-92a6-23ed59971a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.404250 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794de7f-f837-437c-92a6-23ed59971a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.404541 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794de7f-f837-437c-92a6-23ed59971a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.404824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7794de7f-f837-437c-92a6-23ed59971a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.405709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794de7f-f837-437c-92a6-23ed59971a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.406355 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4572e7-25d6-411a-a420-2b964c4f52d5 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.407344 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7794de7f-f837-437c-92a6-23ed59971a47 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.416916 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4572e7-25d6-411a-a420-2b964c4f52d5 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.421082 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1410/2804] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:25:15.421785 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712121d-252a-488c-bd44-68efd199988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.422038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794de7f-f837-437c-92a6-23ed59971a47 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.422446 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712121d-252a-488c-bd44-68efd199988d admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.422737 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712121d-252a-488c-bd44-68efd199988d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.423020 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712121d-252a-488c-bd44-68efd199988d admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.423774 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712121d-252a-488c-bd44-68efd199988d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.425493 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712121d-252a-488c-bd44-68efd199988d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.426822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:15.432060 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7794de7f-f837-437c-92a6-23ed59971a47 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.436881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6da97-6215-4345-904a-1336e41d5c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.437542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6da97-6215-4345-904a-1336e41d5c89 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.437878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6da97-6215-4345-904a-1336e41d5c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.438040 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1395/2805] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:25:15.438158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6da97-6215-4345-904a-1336e41d5c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.438856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6da97-6215-4345-904a-1336e41d5c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.440523 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa6da97-6215-4345-904a-1336e41d5c89 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.441230 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712121d-252a-488c-bd44-68efd199988d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.450566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712121d-252a-488c-bd44-68efd199988d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.450930 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1411/2806] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:25:15.454849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6da97-6215-4345-904a-1336e41d5c89 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.455921 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ad764-1206-454a-921d-6c5db0b285a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.456568 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ad764-1206-454a-921d-6c5db0b285a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.456852 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ad764-1206-454a-921d-6c5db0b285a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.457127 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8ad764-1206-454a-921d-6c5db0b285a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.457845 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ad764-1206-454a-921d-6c5db0b285a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.459485 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8ad764-1206-454a-921d-6c5db0b285a2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.464968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6da97-6215-4345-904a-1336e41d5c89 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.470147 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1396/2807] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:25:15.470287 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eef8ec-dc18-4519-bde6-ed0fc4ed1d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.471737 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eef8ec-dc18-4519-bde6-ed0fc4ed1d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.472294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eef8ec-dc18-4519-bde6-ed0fc4ed1d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.472583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eef8ec-dc18-4519-bde6-ed0fc4ed1d50 admin admin] PATH_INFO: `/v3/projects/b7912e80519d48819e6b38734362fb44/users/17e8f4d1ed02453fa6392db04f8b8f5c/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.474594 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ad764-1206-454a-921d-6c5db0b285a2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.484183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8ad764-1206-454a-921d-6c5db0b285a2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.487782 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eef8ec-dc18-4519-bde6-ed0fc4ed1d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7912e80519d48819e6b38734362fb44, user_id=17e8f4d1ed02453fa6392db04f8b8f5c, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.488965 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210c1f2-86a5-4463-94c1-8e0500bccd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.489887 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210c1f2-86a5-4463-94c1-8e0500bccd83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.490170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210c1f2-86a5-4463-94c1-8e0500bccd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.490446 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210c1f2-86a5-4463-94c1-8e0500bccd83 admin admin] PATH_INFO: `/v3/projects/0d6dd64904a644eeaceb3037f9dbb126/users/541384d1238840c4b2f2b19ad53d91a4/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.490580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eef8ec-dc18-4519-bde6-ed0fc4ed1d50 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.503745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1412/2808] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:15 2022] PUT /identity/v3/projects/b7912e80519d48819e6b38734362fb44/users/17e8f4d1ed02453fa6392db04f8b8f5c/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:25:15.505333 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210c1f2-86a5-4463-94c1-8e0500bccd83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6dd64904a644eeaceb3037f9dbb126, user_id=541384d1238840c4b2f2b19ad53d91a4, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.508008 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210c1f2-86a5-4463-94c1-8e0500bccd83 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.522534 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1397/2809] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:15 2022] PUT /identity/v3/projects/0d6dd64904a644eeaceb3037f9dbb126/users/541384d1238840c4b2f2b19ad53d91a4/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 05 15:25:15.534831 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9669bc5-1c18-4e18-b83c-3bfb8312b1f0 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.535158 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9669bc5-1c18-4e18-b83c-3bfb8312b1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.537122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9669bc5-1c18-4e18-b83c-3bfb8312b1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.564006 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d9669bc5-1c18-4e18-b83c-3bfb8312b1f0 None None] MFA Rules not processed for user `541384d1238840c4b2f2b19ad53d91a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:15.565286 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d9669bc5-1c18-4e18-b83c-3bfb8312b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:15.569418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95919-3f9c-4c67-ba2f-0bfbf223bc28 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.569718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95919-3f9c-4c67-ba2f-0bfbf223bc28 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.570007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95919-3f9c-4c67-ba2f-0bfbf223bc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.601291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-bba95919-3f9c-4c67-ba2f-0bfbf223bc28 None None] MFA Rules not processed for user `17e8f4d1ed02453fa6392db04f8b8f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:15.602513 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bba95919-3f9c-4c67-ba2f-0bfbf223bc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:15.640972 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1413/2810] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:15.653537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9669bc5-1c18-4e18-b83c-3bfb8312b1f0 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.655326 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d9669bc5-1c18-4e18-b83c-3bfb8312b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:15.674971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1398/2811] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:25:15.688350 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95919-3f9c-4c67-ba2f-0bfbf223bc28 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.690405 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bba95919-3f9c-4c67-ba2f-0bfbf223bc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:15.726213 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9669bc5-1c18-4e18-b83c-3bfb8312b1f0 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.730105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9b8f3-7259-44f2-bbbc-75ec5f713e6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.730864 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9b8f3-7259-44f2-bbbc-75ec5f713e6c service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.731288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9b8f3-7259-44f2-bbbc-75ec5f713e6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.731664 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9b8f3-7259-44f2-bbbc-75ec5f713e6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.740462 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9b8f3-7259-44f2-bbbc-75ec5f713e6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.741908 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9b8f3-7259-44f2-bbbc-75ec5f713e6c service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.750856 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1414/2812] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:25:15.758553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95919-3f9c-4c67-ba2f-0bfbf223bc28 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.762214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8bd1b-c35d-44f1-a826-045922be6906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.762866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8bd1b-c35d-44f1-a826-045922be6906 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.763182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8bd1b-c35d-44f1-a826-045922be6906 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.763460 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8bd1b-c35d-44f1-a826-045922be6906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.772589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8bd1b-c35d-44f1-a826-045922be6906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.774027 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8bd1b-c35d-44f1-a826-045922be6906 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.782804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1399/2813] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:25:15.802316 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9b8f3-7259-44f2-bbbc-75ec5f713e6c service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:15.845521 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:15.867182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9b8f3-7259-44f2-bbbc-75ec5f713e6c service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:15.872139 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:15.872992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.873285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.873708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] PATH_INFO: `/v3/users/17e8f4d1ed02453fa6392db04f8b8f5c` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.874956 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e8f4d1ed02453fa6392db04f8b8f5c)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:15.876988 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:15.923100 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e8f4d1ed02453fa6392db04f8b8f5c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.923767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e8f4d1ed02453fa6392db04f8b8f5c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.924425 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e8f4d1ed02453fa6392db04f8b8f5c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.931793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e8f4d1ed02453fa6392db04f8b8f5c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.939075 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e8f4d1ed02453fa6392db04f8b8f5c'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:15.961704 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1415/2814] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:15 2022] DELETE /identity/v3/users/17e8f4d1ed02453fa6392db04f8b8f5c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:15.966170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50563b71-37e0-4ad7-ba00-6c58b0c1aa18 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:15.966464 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50563b71-37e0-4ad7-ba00-6c58b0c1aa18 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:15.966874 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50563b71-37e0-4ad7-ba00-6c58b0c1aa18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:15.990770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-50563b71-37e0-4ad7-ba00-6c58b0c1aa18 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:15.992244 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-50563b71-37e0-4ad7-ba00-6c58b0c1aa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:16.042918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:16.062943 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1400/2815] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:16.070016 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:16.071555 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:16.131143 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cbc2c9-d4e9-4641-a05b-a8fab6b8945b admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:16.134781 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fb9ae-59e4-4ce2-ac16-61012252eef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:16.135459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fb9ae-59e4-4ce2-ac16-61012252eef8 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.135746 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fb9ae-59e4-4ce2-ac16-61012252eef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.136026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fb9ae-59e4-4ce2-ac16-61012252eef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:16.144878 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fb9ae-59e4-4ce2-ac16-61012252eef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:16.146302 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79fb9ae-59e4-4ce2-ac16-61012252eef8 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:16.155603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1416/2816] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:16.438536 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50563b71-37e0-4ad7-ba00-6c58b0c1aa18 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:16.439996 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-50563b71-37e0-4ad7-ba00-6c58b0c1aa18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:16.452156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50563b71-37e0-4ad7-ba00-6c58b0c1aa18 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:16.456838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:16.457700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.457990 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.458268 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] PATH_INFO: `/v3/projects/b7912e80519d48819e6b38734362fb44` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:16.459538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7912e80519d48819e6b38734362fb44)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:16.461439 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:16.507688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7912e80519d48819e6b38734362fb44', 'request_id': 'req-9d4e2384-977b-4899-9f10-ad188cfb97eb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:16.516123 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7912e80519d48819e6b38734362fb44', 'request_id': 'req-9d4e2384-977b-4899-9f10-ad188cfb97eb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:16.523538 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7912e80519d48819e6b38734362fb44', 'request_id': 'req-9d4e2384-977b-4899-9f10-ad188cfb97eb'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:16.534338 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] The token cache is being invalidate because project b7912e80519d48819e6b38734362fb44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:16.535165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:16.535804 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-9d4e2384-977b-4899-9f10-ad188cfb97eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:16.537332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1401/2817] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:16 2022] DELETE /identity/v3/projects/b7912e80519d48819e6b38734362fb44 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:16.542122 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56b37b-4db5-46da-8c64-98d6b9394f17 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.542605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56b37b-4db5-46da-8c64-98d6b9394f17 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.542722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56b37b-4db5-46da-8c64-98d6b9394f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:16.560406 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-8c56b37b-4db5-46da-8c64-98d6b9394f17 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:16.561603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8c56b37b-4db5-46da-8c64-98d6b9394f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:16.619656 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1417/2818] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:16.626971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec89d4b-431f-4138-b77c-c91dd7c2bbd0 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.627305 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec89d4b-431f-4138-b77c-c91dd7c2bbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.627588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec89d4b-431f-4138-b77c-c91dd7c2bbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:16.645332 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-4ec89d4b-431f-4138-b77c-c91dd7c2bbd0 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:16.646526 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-4ec89d4b-431f-4138-b77c-c91dd7c2bbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:16.654254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1402/2819] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:16.658134 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56b37b-4db5-46da-8c64-98d6b9394f17 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:16.663552 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:16.670011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56b37b-4db5-46da-8c64-98d6b9394f17 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:16.675964 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d85d8-b3b7-481b-b1e7-bb8278ff08ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:16.676624 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d85d8-b3b7-481b-b1e7-bb8278ff08ed admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.676909 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d85d8-b3b7-481b-b1e7-bb8278ff08ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.677191 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d85d8-b3b7-481b-b1e7-bb8278ff08ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:16.677913 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d85d8-b3b7-481b-b1e7-bb8278ff08ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:25:16.678230 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d85d8-b3b7-481b-b1e7-bb8278ff08ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:16.679877 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d85d8-b3b7-481b-b1e7-bb8278ff08ed admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:16.691335 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1418/2820] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:25:16.695547 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec89d4b-431f-4138-b77c-c91dd7c2bbd0 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:16.704997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec89d4b-431f-4138-b77c-c91dd7c2bbd0 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:16.709675 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3604-165d-4c8e-b13e-3d918f38b85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:16.710329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3604-165d-4c8e-b13e-3d918f38b85d admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.710623 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3604-165d-4c8e-b13e-3d918f38b85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.710905 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98b3604-165d-4c8e-b13e-3d918f38b85d admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:16.711639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b3604-165d-4c8e-b13e-3d918f38b85d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:16.713575 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98b3604-165d-4c8e-b13e-3d918f38b85d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:16.740758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1403/2821] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:16 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:16.744953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d85d8-b3b7-481b-b1e7-bb8278ff08ed admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:16.756428 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d85d8-b3b7-481b-b1e7-bb8278ff08ed admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:16.819365 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d6d9e-51fa-4c59-8a25-8601fff49b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:16.820109 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d6d9e-51fa-4c59-8a25-8601fff49b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.820470 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d6d9e-51fa-4c59-8a25-8601fff49b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.820832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d6d9e-51fa-4c59-8a25-8601fff49b53 admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:16.821687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d6d9e-51fa-4c59-8a25-8601fff49b53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:16.823894 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d6d9e-51fa-4c59-8a25-8601fff49b53 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:16.848095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1419/2822] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:16 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:16.852205 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3604-165d-4c8e-b13e-3d918f38b85d admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:16.862322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98b3604-165d-4c8e-b13e-3d918f38b85d admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:16.867004 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d8fca-ca1c-48f7-a50f-91328547f9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:16.867690 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d8fca-ca1c-48f7-a50f-91328547f9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.867971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d8fca-ca1c-48f7-a50f-91328547f9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.868242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d8fca-ca1c-48f7-a50f-91328547f9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:16.868994 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d8fca-ca1c-48f7-a50f-91328547f9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:16.870589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d8fca-ca1c-48f7-a50f-91328547f9f2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:16.882221 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1404/2823] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:16.886281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d6d9e-51fa-4c59-8a25-8601fff49b53 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:16.897232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d6d9e-51fa-4c59-8a25-8601fff49b53 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:16.901979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dff5bc-c076-4d83-a243-34893355d881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:16.902670 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dff5bc-c076-4d83-a243-34893355d881 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.902962 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dff5bc-c076-4d83-a243-34893355d881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.903304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dff5bc-c076-4d83-a243-34893355d881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:16.904450 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dff5bc-c076-4d83-a243-34893355d881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:16.905679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dff5bc-c076-4d83-a243-34893355d881 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:16.916496 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1420/2824] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:16.920549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d8fca-ca1c-48f7-a50f-91328547f9f2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:16.930340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d8fca-ca1c-48f7-a50f-91328547f9f2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:16.934914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a24a84-3237-49c7-bd89-35c6b1991296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:16.935828 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24a84-3237-49c7-bd89-35c6b1991296 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.936113 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24a84-3237-49c7-bd89-35c6b1991296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.936389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a24a84-3237-49c7-bd89-35c6b1991296 admin admin] PATH_INFO: `/v3/projects/1fbea6a34df340f2ad0d3eb6728a87b2/users/3edc5702286d4e25a294f2e3c217a0d1/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:16.950559 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a24a84-3237-49c7-bd89-35c6b1991296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fbea6a34df340f2ad0d3eb6728a87b2, user_id=3edc5702286d4e25a294f2e3c217a0d1, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:16.953237 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a24a84-3237-49c7-bd89-35c6b1991296 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:16.965993 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1405/2825] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:16 2022] PUT /identity/v3/projects/1fbea6a34df340f2ad0d3eb6728a87b2/users/3edc5702286d4e25a294f2e3c217a0d1/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:16.977102 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a9bc4-2d3e-412b-8071-0a009bf9aac1 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:16.977401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a9bc4-2d3e-412b-8071-0a009bf9aac1 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:16.977684 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a9bc4-2d3e-412b-8071-0a009bf9aac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:17.003683 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-1c0a9bc4-2d3e-412b-8071-0a009bf9aac1 None None] MFA Rules not processed for user `3edc5702286d4e25a294f2e3c217a0d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:17.004888 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-1c0a9bc4-2d3e-412b-8071-0a009bf9aac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:17.069578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1421/2826] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:17.079918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a24a84-3237-49c7-bd89-35c6b1991296 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:17.082242 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-05a24a84-3237-49c7-bd89-35c6b1991296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:17.143217 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a24a84-3237-49c7-bd89-35c6b1991296 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:17.147529 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fa292-b3b6-48dc-a342-cc2b790aa988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:17.148170 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fa292-b3b6-48dc-a342-cc2b790aa988 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:17.148458 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fa292-b3b6-48dc-a342-cc2b790aa988 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:17.148736 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fa292-b3b6-48dc-a342-cc2b790aa988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:17.157151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fa292-b3b6-48dc-a342-cc2b790aa988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:17.158665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fa292-b3b6-48dc-a342-cc2b790aa988 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:17.168074 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1406/2827] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:27.065574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:27.163951 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:33.177727 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a9bc4-2d3e-412b-8071-0a009bf9aac1 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:33.179220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-1c0a9bc4-2d3e-412b-8071-0a009bf9aac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:33.242291 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a9bc4-2d3e-412b-8071-0a009bf9aac1 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:33.246089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ad86d-0c14-4ec0-82f4-869d00bd0638 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:33.246820 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ad86d-0c14-4ec0-82f4-869d00bd0638 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:33.247147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ad86d-0c14-4ec0-82f4-869d00bd0638 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:33.247431 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ad86d-0c14-4ec0-82f4-869d00bd0638 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:33.249362 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-9c0ad86d-0c14-4ec0-82f4-869d00bd0638 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:33.259411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ad86d-0c14-4ec0-82f4-869d00bd0638 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:33.260927 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0ad86d-0c14-4ec0-82f4-869d00bd0638 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:33.269872 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1422/2828] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:35.926322 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fa292-b3b6-48dc-a342-cc2b790aa988 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:35.986539 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fa292-b3b6-48dc-a342-cc2b790aa988 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:35.991878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:35.992799 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:35.993096 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:35.993374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] PATH_INFO: `/v3/users/3edc5702286d4e25a294f2e3c217a0d1` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:35.994567 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3edc5702286d4e25a294f2e3c217a0d1)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:35.996516 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:36.041584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3edc5702286d4e25a294f2e3c217a0d1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:36.042233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3edc5702286d4e25a294f2e3c217a0d1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:36.042841 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3edc5702286d4e25a294f2e3c217a0d1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:36.050987 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3edc5702286d4e25a294f2e3c217a0d1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:36.057884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3edc5702286d4e25a294f2e3c217a0d1'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:36.075650 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1407/2829] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:35 2022] DELETE /identity/v3/users/3edc5702286d4e25a294f2e3c217a0d1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:36.080285 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c02f7-81ee-4701-8a33-71cc37a994dc None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:36.080760 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c02f7-81ee-4701-8a33-71cc37a994dc None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:36.080868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1c02f7-81ee-4701-8a33-71cc37a994dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:36.098181 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ad1c02f7-81ee-4701-8a33-71cc37a994dc None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:36.099399 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ad1c02f7-81ee-4701-8a33-71cc37a994dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:36.154922 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1423/2830] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:36.164827 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:36.166203 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:36.222695 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34209c-88f5-425c-81fb-33c2dc5482e0 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:36.226312 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92123c44-63e1-4c24-984e-b2aae54f8300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:36.226952 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92123c44-63e1-4c24-984e-b2aae54f8300 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:36.227266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92123c44-63e1-4c24-984e-b2aae54f8300 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:36.227544 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92123c44-63e1-4c24-984e-b2aae54f8300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:36.236374 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92123c44-63e1-4c24-984e-b2aae54f8300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:36.237798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92123c44-63e1-4c24-984e-b2aae54f8300 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:36.247716 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1408/2831] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:36.516849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c02f7-81ee-4701-8a33-71cc37a994dc None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:36.518970 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ad1c02f7-81ee-4701-8a33-71cc37a994dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:36.534232 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1c02f7-81ee-4701-8a33-71cc37a994dc None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:36.540492 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:36.541329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:36.541641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:36.541994 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] PATH_INFO: `/v3/projects/1fbea6a34df340f2ad0d3eb6728a87b2` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:36.543090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fbea6a34df340f2ad0d3eb6728a87b2)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:36.545026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:36.590639 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fbea6a34df340f2ad0d3eb6728a87b2', 'request_id': 'req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:36.599104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fbea6a34df340f2ad0d3eb6728a87b2', 'request_id': 'req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:36.605195 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fbea6a34df340f2ad0d3eb6728a87b2', 'request_id': 'req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:36.615776 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] The token cache is being invalidate because project 1fbea6a34df340f2ad0d3eb6728a87b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:36.616114 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:36.616706 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:36.618266 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1424/2832] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:36 2022] DELETE /identity/v3/projects/1fbea6a34df340f2ad0d3eb6728a87b2 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:40.431628 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92123c44-63e1-4c24-984e-b2aae54f8300 service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:40.433295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-92123c44-63e1-4c24-984e-b2aae54f8300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:40.461480 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:40.510145 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92123c44-63e1-4c24-984e-b2aae54f8300 service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:40.572295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20473199-a742-4311-ac40-0171bd41cb40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:40.572876 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20473199-a742-4311-ac40-0171bd41cb40 service nova] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:40.573191 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20473199-a742-4311-ac40-0171bd41cb40 service nova] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:40.573473 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20473199-a742-4311-ac40-0171bd41cb40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:40.575274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-20473199-a742-4311-ac40-0171bd41cb40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:40.587647 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20473199-a742-4311-ac40-0171bd41cb40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:40.589214 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20473199-a742-4311-ac40-0171bd41cb40 service nova] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:40.599743 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1409/2833] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 05 15:25:45.614127 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:45.615535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:45.644766 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:45.689059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578075-2dbc-4af6-afb9-2dfd43f8e5b0 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:45.694217 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:45.695118 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:45.695411 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:45.695691 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] PATH_INFO: `/v3/users/541384d1238840c4b2f2b19ad53d91a4` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:45.696891 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=541384d1238840c4b2f2b19ad53d91a4)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:45.698825 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:45.743787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '541384d1238840c4b2f2b19ad53d91a4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:45.744444 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '541384d1238840c4b2f2b19ad53d91a4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:45.745029 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '541384d1238840c4b2f2b19ad53d91a4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:45.751710 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '541384d1238840c4b2f2b19ad53d91a4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:45.758822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '541384d1238840c4b2f2b19ad53d91a4'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:45.776238 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1425/2834] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:25:45 2022] DELETE /identity/v3/users/541384d1238840c4b2f2b19ad53d91a4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:45.780776 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c09209-1b65-407a-8f06-1898b59fc049 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:45.781270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c09209-1b65-407a-8f06-1898b59fc049 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:45.781389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c09209-1b65-407a-8f06-1898b59fc049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:45.799033 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-45c09209-1b65-407a-8f06-1898b59fc049 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:45.800271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-45c09209-1b65-407a-8f06-1898b59fc049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:45.859329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1410/2835] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:45.868817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:45.870254 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:45.926546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7a21f-7f79-4d24-9ce6-f79218e79f4d admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:45.930144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2724bc-fcd3-4eb5-b2da-7b749ca4b110 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:45.930785 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2724bc-fcd3-4eb5-b2da-7b749ca4b110 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:45.931095 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2724bc-fcd3-4eb5-b2da-7b749ca4b110 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:45.931379 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2724bc-fcd3-4eb5-b2da-7b749ca4b110 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:45.940526 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2724bc-fcd3-4eb5-b2da-7b749ca4b110 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:45.942488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2724bc-fcd3-4eb5-b2da-7b749ca4b110 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:45.951811 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1426/2836] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:46.221838 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c09209-1b65-407a-8f06-1898b59fc049 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:46.223282 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-45c09209-1b65-407a-8f06-1898b59fc049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:46.235156 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c09209-1b65-407a-8f06-1898b59fc049 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:46.240549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:46.241411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.241688 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.241960 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] PATH_INFO: `/v3/projects/0d6dd64904a644eeaceb3037f9dbb126` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.243172 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6dd64904a644eeaceb3037f9dbb126)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:46.245073 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:46.286271 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:46.292922 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6dd64904a644eeaceb3037f9dbb126', 'request_id': 'req-96402f33-6abc-4189-8682-98c243f72574'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:46.299944 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6dd64904a644eeaceb3037f9dbb126', 'request_id': 'req-96402f33-6abc-4189-8682-98c243f72574'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:46.307334 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6dd64904a644eeaceb3037f9dbb126', 'request_id': 'req-96402f33-6abc-4189-8682-98c243f72574'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:46.318266 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] The token cache is being invalidate because project 0d6dd64904a644eeaceb3037f9dbb126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:25:46.318558 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:46.319233 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-96402f33-6abc-4189-8682-98c243f72574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:25:46.320762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1411/2837] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:25:46 2022] DELETE /identity/v3/projects/0d6dd64904a644eeaceb3037f9dbb126 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:46.325641 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7afec-aac5-4dcc-8f83-8b0351a675ff None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.325932 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7afec-aac5-4dcc-8f83-8b0351a675ff None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.326222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7afec-aac5-4dcc-8f83-8b0351a675ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.343604 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-0eb7afec-aac5-4dcc-8f83-8b0351a675ff None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:46.344799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-0eb7afec-aac5-4dcc-8f83-8b0351a675ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:46.377902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:46.405090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1427/2838] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:46.413744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86661a56-29bc-4926-84b2-8c066ea58bd5 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.414021 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86661a56-29bc-4926-84b2-8c066ea58bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.414302 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86661a56-29bc-4926-84b2-8c066ea58bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.433521 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-86661a56-29bc-4926-84b2-8c066ea58bd5 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:46.434725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-86661a56-29bc-4926-84b2-8c066ea58bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:46.442555 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1412/2839] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:46.446640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7afec-aac5-4dcc-8f83-8b0351a675ff None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:46.456289 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7afec-aac5-4dcc-8f83-8b0351a675ff None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:46.460943 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b091c-afea-4260-a634-cebccaf8fb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:46.461582 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b091c-afea-4260-a634-cebccaf8fb45 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.461868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b091c-afea-4260-a634-cebccaf8fb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.462150 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b091c-afea-4260-a634-cebccaf8fb45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.462881 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b091c-afea-4260-a634-cebccaf8fb45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80351) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:25:46.463229 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b091c-afea-4260-a634-cebccaf8fb45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:46.464865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b091c-afea-4260-a634-cebccaf8fb45 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:46.477000 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1428/2840] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:25:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:25:46.481300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86661a56-29bc-4926-84b2-8c066ea58bd5 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:46.491097 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86661a56-29bc-4926-84b2-8c066ea58bd5 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:46.495878 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d10e6-6d29-4aaa-b1b4-1e57e96d7328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:46.496529 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d10e6-6d29-4aaa-b1b4-1e57e96d7328 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.496813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d10e6-6d29-4aaa-b1b4-1e57e96d7328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.497086 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d10e6-6d29-4aaa-b1b4-1e57e96d7328 admin admin] PATH_INFO: `/v3/projects` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.497770 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d10e6-6d29-4aaa-b1b4-1e57e96d7328 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:46.499758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d10e6-6d29-4aaa-b1b4-1e57e96d7328 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:46.525359 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1413/2841] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:25:46 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:46.529288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b091c-afea-4260-a634-cebccaf8fb45 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:46.539666 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b091c-afea-4260-a634-cebccaf8fb45 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:46.594091 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed9a1e-247c-4d9f-bcdd-89de23737c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:46.594679 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed9a1e-247c-4d9f-bcdd-89de23737c4e admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.594968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed9a1e-247c-4d9f-bcdd-89de23737c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.595283 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed9a1e-247c-4d9f-bcdd-89de23737c4e admin admin] PATH_INFO: `/v3/users` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.595980 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed9a1e-247c-4d9f-bcdd-89de23737c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:46.597904 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed9a1e-247c-4d9f-bcdd-89de23737c4e admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:46.618599 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1429/2842] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:25:46 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:25:46.622626 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d10e6-6d29-4aaa-b1b4-1e57e96d7328 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:46.632913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d10e6-6d29-4aaa-b1b4-1e57e96d7328 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:46.637476 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0006dc7-4739-42ac-8673-c8f8c04ac62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:46.638133 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0006dc7-4739-42ac-8673-c8f8c04ac62a admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.638421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0006dc7-4739-42ac-8673-c8f8c04ac62a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.638701 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0006dc7-4739-42ac-8673-c8f8c04ac62a admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.639466 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0006dc7-4739-42ac-8673-c8f8c04ac62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:46.641077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0006dc7-4739-42ac-8673-c8f8c04ac62a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:46.652389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1414/2843] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:25:46.656032 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed9a1e-247c-4d9f-bcdd-89de23737c4e admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:46.666297 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed9a1e-247c-4d9f-bcdd-89de23737c4e admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:46.670849 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f66ae-881c-408a-af6f-2976837cab2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:46.671537 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f66ae-881c-408a-af6f-2976837cab2c admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.671822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f66ae-881c-408a-af6f-2976837cab2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.672106 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f66ae-881c-408a-af6f-2976837cab2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.672827 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f66ae-881c-408a-af6f-2976837cab2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:46.674409 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f66ae-881c-408a-af6f-2976837cab2c admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:46.684742 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1430/2844] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:25:46.688406 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0006dc7-4739-42ac-8673-c8f8c04ac62a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:46.698137 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0006dc7-4739-42ac-8673-c8f8c04ac62a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:46.702747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cc0be-3c24-422e-ad9f-5be8d952ea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:46.703651 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cc0be-3c24-422e-ad9f-5be8d952ea94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.703946 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cc0be-3c24-422e-ad9f-5be8d952ea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.704228 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cc0be-3c24-422e-ad9f-5be8d952ea94 admin admin] PATH_INFO: `/v3/projects/23753da7e9ec4cd0b9a6a1c7d348d427/users/8eedaeaa2346442ea77007d61f1563d3/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.724689 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cc0be-3c24-422e-ad9f-5be8d952ea94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23753da7e9ec4cd0b9a6a1c7d348d427, user_id=8eedaeaa2346442ea77007d61f1563d3, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:46.729051 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70cc0be-3c24-422e-ad9f-5be8d952ea94 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:46.744832 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1415/2845] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:25:46 2022] PUT /identity/v3/projects/23753da7e9ec4cd0b9a6a1c7d348d427/users/8eedaeaa2346442ea77007d61f1563d3/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:25:46.817275 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4d970-0017-4f6d-9d9f-629f0d8ba74e None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.817572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4d970-0017-4f6d-9d9f-629f0d8ba74e None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.817854 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4d970-0017-4f6d-9d9f-629f0d8ba74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.844627 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-bbf4d970-0017-4f6d-9d9f-629f0d8ba74e None None] MFA Rules not processed for user `8eedaeaa2346442ea77007d61f1563d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:25:46.845765 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bbf4d970-0017-4f6d-9d9f-629f0d8ba74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:46.907836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1431/2846] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:25:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:25:46.919574 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cc0be-3c24-422e-ad9f-5be8d952ea94 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:25:46.921313 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b70cc0be-3c24-422e-ad9f-5be8d952ea94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:25:46.977254 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70cc0be-3c24-422e-ad9f-5be8d952ea94 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:25:46.980844 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d31c8-c2e8-45ca-989c-5d9e1a2e33f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:25:46.981472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d31c8-c2e8-45ca-989c-5d9e1a2e33f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:25:46.981757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d31c8-c2e8-45ca-989c-5d9e1a2e33f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:25:46.982037 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d31c8-c2e8-45ca-989c-5d9e1a2e33f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:25:46.990195 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d31c8-c2e8-45ca-989c-5d9e1a2e33f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:25:46.991758 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d31c8-c2e8-45ca-989c-5d9e1a2e33f2 service cinder] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:25:47.000522 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1416/2847] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:25:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:25:56.904954 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:25:56.996411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=62,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:13.620401 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4d970-0017-4f6d-9d9f-629f0d8ba74e None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:13.621824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bbf4d970-0017-4f6d-9d9f-629f0d8ba74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:13.741025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4d970-0017-4f6d-9d9f-629f0d8ba74e None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:13.746649 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55b674-4b07-4213-94fc-f4601d76d9de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:13.747331 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55b674-4b07-4213-94fc-f4601d76d9de service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:13.747621 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55b674-4b07-4213-94fc-f4601d76d9de service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:13.747902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55b674-4b07-4213-94fc-f4601d76d9de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:13.749676 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-8d55b674-4b07-4213-94fc-f4601d76d9de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:13.762007 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55b674-4b07-4213-94fc-f4601d76d9de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:13.763661 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55b674-4b07-4213-94fc-f4601d76d9de service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:13.774396 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1432/2848] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:17.256216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d31c8-c2e8-45ca-989c-5d9e1a2e33f2 service cinder] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:17.257807 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0f9d31c8-c2e8-45ca-989c-5d9e1a2e33f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:17.358775 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d31c8-c2e8-45ca-989c-5d9e1a2e33f2 service cinder] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:17.364200 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4c885-32e1-4ef3-9028-8a9aa0a95cdf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:17.365004 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c885-32e1-4ef3-9028-8a9aa0a95cdf service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:17.365318 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c885-32e1-4ef3-9028-8a9aa0a95cdf service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:17.365766 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae4c885-32e1-4ef3-9028-8a9aa0a95cdf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:17.367619 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6ae4c885-32e1-4ef3-9028-8a9aa0a95cdf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:17.397817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4c885-32e1-4ef3-9028-8a9aa0a95cdf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:17.397817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae4c885-32e1-4ef3-9028-8a9aa0a95cdf service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:17.403636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1417/2849] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:23.416723 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55b674-4b07-4213-94fc-f4601d76d9de service nova] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:23.451180 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55b674-4b07-4213-94fc-f4601d76d9de service nova] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:23.457144 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb933efb-372a-41e9-9bf7-aba729cf57e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7a416f22d33450183801cef2b13822f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:23.457929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb933efb-372a-41e9-9bf7-aba729cf57e7 service nova] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:23.458303 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb933efb-372a-41e9-9bf7-aba729cf57e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:23.458646 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb933efb-372a-41e9-9bf7-aba729cf57e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:23.460652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-eb933efb-372a-41e9-9bf7-aba729cf57e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:23.489824 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb933efb-372a-41e9-9bf7-aba729cf57e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:23.491563 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb933efb-372a-41e9-9bf7-aba729cf57e7 service nova] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:23.503233 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1433/2850] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:26.066898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4c885-32e1-4ef3-9028-8a9aa0a95cdf service placement] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:26.068636 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-6ae4c885-32e1-4ef3-9028-8a9aa0a95cdf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:26.144800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae4c885-32e1-4ef3-9028-8a9aa0a95cdf service placement] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:26.148369 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60de2c-b693-4d48-8f7b-fc4b7c81cc7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:26.149006 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60de2c-b693-4d48-8f7b-fc4b7c81cc7b service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:26.149289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60de2c-b693-4d48-8f7b-fc4b7c81cc7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:26.149566 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60de2c-b693-4d48-8f7b-fc4b7c81cc7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:26.151880 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-ce60de2c-b693-4d48-8f7b-fc4b7c81cc7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:26.212197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=100 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:26.234797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60de2c-b693-4d48-8f7b-fc4b7c81cc7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:26.236289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60de2c-b693-4d48-8f7b-fc4b7c81cc7b service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:26.254639 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1418/2851] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:29.405948 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb933efb-372a-41e9-9bf7-aba729cf57e7 service nova] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:29.418505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb933efb-372a-41e9-9bf7-aba729cf57e7 service nova] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:29.423931 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790186a-f36e-40b7-be5c-392824171006 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:29.424686 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790186a-f36e-40b7-be5c-392824171006 service placement] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:29.425002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790186a-f36e-40b7-be5c-392824171006 service placement] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:29.425288 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790186a-f36e-40b7-be5c-392824171006 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:29.427119 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f790186a-f36e-40b7-be5c-392824171006 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:29.443452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790186a-f36e-40b7-be5c-392824171006 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:29.455300 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790186a-f36e-40b7-be5c-392824171006 service placement] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:29.464465 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1434/2852] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:30.239986 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60de2c-b693-4d48-8f7b-fc4b7c81cc7b service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:30.250346 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60de2c-b693-4d48-8f7b-fc4b7c81cc7b service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:30.256800 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:30.257720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:30.258007 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:30.258289 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] PATH_INFO: `/v3/users/8eedaeaa2346442ea77007d61f1563d3` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:30.259576 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eedaeaa2346442ea77007d61f1563d3)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:30.261604 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:30.321997 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eedaeaa2346442ea77007d61f1563d3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:30.323329 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eedaeaa2346442ea77007d61f1563d3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:30.323958 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eedaeaa2346442ea77007d61f1563d3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:30.332778 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eedaeaa2346442ea77007d61f1563d3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:30.343839 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eedaeaa2346442ea77007d61f1563d3'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:30.365390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1419/2853] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:26:30 2022] DELETE /identity/v3/users/8eedaeaa2346442ea77007d61f1563d3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:26:30.370078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1f1b-5bb4-44f3-86d8-ed429be663d9 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:30.370375 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1f1b-5bb4-44f3-86d8-ed429be663d9 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:30.370665 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1f1b-5bb4-44f3-86d8-ed429be663d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:30.401069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-d5bc1f1b-5bb4-44f3-86d8-ed429be663d9 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:30.402466 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d5bc1f1b-5bb4-44f3-86d8-ed429be663d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:30.456907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=100 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:30.479026 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1435/2854] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:30.488747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:30.490583 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:30.552991 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c00e333-b4f4-4f14-8410-eeed6b4ba86f admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:30.568801 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202f05c-030b-48b1-a182-4e4a36bb08f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:30.569597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202f05c-030b-48b1-a182-4e4a36bb08f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:30.569982 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202f05c-030b-48b1-a182-4e4a36bb08f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:30.570541 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202f05c-030b-48b1-a182-4e4a36bb08f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:30.580817 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202f05c-030b-48b1-a182-4e4a36bb08f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:30.582425 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202f05c-030b-48b1-a182-4e4a36bb08f8 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:30.594389 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1420/2855] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:31.008516 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc1f1b-5bb4-44f3-86d8-ed429be663d9 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:31.010903 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-d5bc1f1b-5bb4-44f3-86d8-ed429be663d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:31.023816 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc1f1b-5bb4-44f3-86d8-ed429be663d9 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:31.031804 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:31.033146 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.033884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.034493 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] PATH_INFO: `/v3/projects/23753da7e9ec4cd0b9a6a1c7d348d427` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.037177 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] RBAC: Authorizing `identity:delete_project(project_id=23753da7e9ec4cd0b9a6a1c7d348d427)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:31.039645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:31.108866 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23753da7e9ec4cd0b9a6a1c7d348d427', 'request_id': 'req-85cf8574-fd89-4d95-9081-35065d04f98b'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:31.119808 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23753da7e9ec4cd0b9a6a1c7d348d427', 'request_id': 'req-85cf8574-fd89-4d95-9081-35065d04f98b'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:31.129658 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23753da7e9ec4cd0b9a6a1c7d348d427', 'request_id': 'req-85cf8574-fd89-4d95-9081-35065d04f98b'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:31.146036 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] The token cache is being invalidate because project 23753da7e9ec4cd0b9a6a1c7d348d427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:26:31.146898 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:31.147165 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-85cf8574-fd89-4d95-9081-35065d04f98b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:31.152308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1436/2856] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:26:31 2022] DELETE /identity/v3/projects/23753da7e9ec4cd0b9a6a1c7d348d427 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:26:31.153822 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580dfe5-2d21-4f9b-86d4-0a16fdacd5a1 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.154116 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580dfe5-2d21-4f9b-86d4-0a16fdacd5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.154395 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580dfe5-2d21-4f9b-86d4-0a16fdacd5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.215652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-0580dfe5-2d21-4f9b-86d4-0a16fdacd5a1 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:31.217263 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0580dfe5-2d21-4f9b-86d4-0a16fdacd5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:31.230957 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:31.287884 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1421/2857] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:31.295953 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe94dd-4e77-47ce-b2d6-8d509933de07 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.296248 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe94dd-4e77-47ce-b2d6-8d509933de07 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.296817 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe94dd-4e77-47ce-b2d6-8d509933de07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.319168 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-7cbe94dd-4e77-47ce-b2d6-8d509933de07 None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:31.320125 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7cbe94dd-4e77-47ce-b2d6-8d509933de07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:31.331622 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1437/2858] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:31.333089 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580dfe5-2d21-4f9b-86d4-0a16fdacd5a1 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:31.344992 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580dfe5-2d21-4f9b-86d4-0a16fdacd5a1 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:31.349981 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9adf69-34c1-4355-85d1-b88022c989fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:31.350663 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9adf69-34c1-4355-85d1-b88022c989fe admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.350971 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9adf69-34c1-4355-85d1-b88022c989fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.351295 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9adf69-34c1-4355-85d1-b88022c989fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.352014 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9adf69-34c1-4355-85d1-b88022c989fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:26:31.352330 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9adf69-34c1-4355-85d1-b88022c989fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:31.354549 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9adf69-34c1-4355-85d1-b88022c989fe admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:31.367771 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1422/2859] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:26:31.372557 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe94dd-4e77-47ce-b2d6-8d509933de07 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:31.387625 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe94dd-4e77-47ce-b2d6-8d509933de07 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:31.395025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32739e-7ee8-4163-bc60-12221a9b36a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:31.395733 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32739e-7ee8-4163-bc60-12221a9b36a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.396606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32739e-7ee8-4163-bc60-12221a9b36a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.396900 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32739e-7ee8-4163-bc60-12221a9b36a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.398159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32739e-7ee8-4163-bc60-12221a9b36a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:31.401780 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32739e-7ee8-4163-bc60-12221a9b36a8 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:31.436655 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1438/2860] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:26:31 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:26:31.440849 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9adf69-34c1-4355-85d1-b88022c989fe admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:31.452797 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9adf69-34c1-4355-85d1-b88022c989fe admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:31.516503 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c34c3-4eb8-4811-97b4-23d0b3448d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:31.517252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c34c3-4eb8-4811-97b4-23d0b3448d6a admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.518017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c34c3-4eb8-4811-97b4-23d0b3448d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.518017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8c34c3-4eb8-4811-97b4-23d0b3448d6a admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.518764 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c34c3-4eb8-4811-97b4-23d0b3448d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:31.520936 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8c34c3-4eb8-4811-97b4-23d0b3448d6a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:31.544035 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1423/2861] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:26:31 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 05 15:26:31.548152 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32739e-7ee8-4163-bc60-12221a9b36a8 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:31.560587 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32739e-7ee8-4163-bc60-12221a9b36a8 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:31.575443 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e3689-03f4-4d15-8405-7a05f5a9cb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:31.576873 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e3689-03f4-4d15-8405-7a05f5a9cb99 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.577477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e3689-03f4-4d15-8405-7a05f5a9cb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.577979 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e3689-03f4-4d15-8405-7a05f5a9cb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.578895 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e3689-03f4-4d15-8405-7a05f5a9cb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:31.581025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826e3689-03f4-4d15-8405-7a05f5a9cb99 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:31.593687 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1439/2862] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:26:31.597762 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c34c3-4eb8-4811-97b4-23d0b3448d6a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:31.607873 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8c34c3-4eb8-4811-97b4-23d0b3448d6a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:31.612741 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe96ab-c084-4018-a993-a9f0d78907f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:31.613417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe96ab-c084-4018-a993-a9f0d78907f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.613707 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe96ab-c084-4018-a993-a9f0d78907f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.613989 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe96ab-c084-4018-a993-a9f0d78907f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.614725 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe96ab-c084-4018-a993-a9f0d78907f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:31.616421 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe96ab-c084-4018-a993-a9f0d78907f1 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:31.628648 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1424/2863] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:26:31.632974 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e3689-03f4-4d15-8405-7a05f5a9cb99 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:31.643865 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e3689-03f4-4d15-8405-7a05f5a9cb99 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:31.649077 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac0a67-187b-40e8-b1ab-698a0e07e060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:31.650038 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0a67-187b-40e8-b1ab-698a0e07e060 admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.650330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0a67-187b-40e8-b1ab-698a0e07e060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.650610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac0a67-187b-40e8-b1ab-698a0e07e060 admin admin] PATH_INFO: `/v3/projects/c7984f018652497b8844155091acd2c3/users/c92576b377794d099fb0920e3b973293/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.666329 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac0a67-187b-40e8-b1ab-698a0e07e060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7984f018652497b8844155091acd2c3, user_id=c92576b377794d099fb0920e3b973293, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:31.669052 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac0a67-187b-40e8-b1ab-698a0e07e060 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:31.700101 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1440/2864] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:26:31 2022] PUT /identity/v3/projects/c7984f018652497b8844155091acd2c3/users/c92576b377794d099fb0920e3b973293/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:26:31.704309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b483c5-2851-4780-a99c-559aa58cca0b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.704612 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b483c5-2851-4780-a99c-559aa58cca0b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.704899 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b483c5-2851-4780-a99c-559aa58cca0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.738622 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-54b483c5-2851-4780-a99c-559aa58cca0b None None] MFA Rules not processed for user `c92576b377794d099fb0920e3b973293`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:31.740290 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-54b483c5-2851-4780-a99c-559aa58cca0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:31.841910 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1425/2865] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:31.854098 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac0a67-187b-40e8-b1ab-698a0e07e060 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:31.855960 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-33ac0a67-187b-40e8-b1ab-698a0e07e060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:31.927644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac0a67-187b-40e8-b1ab-698a0e07e060 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:31.931970 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcf94d-7de8-4fbd-b67f-a5fd891944b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:31.932626 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcf94d-7de8-4fbd-b67f-a5fd891944b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:31.933090 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcf94d-7de8-4fbd-b67f-a5fd891944b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:31.933372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcf94d-7de8-4fbd-b67f-a5fd891944b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:31.942182 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcf94d-7de8-4fbd-b67f-a5fd891944b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:31.943709 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fcf94d-7de8-4fbd-b67f-a5fd891944b0 service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:31.953758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1441/2866] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:32.838813 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b483c5-2851-4780-a99c-559aa58cca0b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:32.841579 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-54b483c5-2851-4780-a99c-559aa58cca0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:32.878806 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:32.902278 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcf94d-7de8-4fbd-b67f-a5fd891944b0 service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:32.904957 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-b0fcf94d-7de8-4fbd-b67f-a5fd891944b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:32.921280 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:33.006798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b483c5-2851-4780-a99c-559aa58cca0b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:33.025668 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0910a607-97bb-47ba-83ab-65e810f757c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:33.029218 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910a607-97bb-47ba-83ab-65e810f757c1 service placement] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:33.029506 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910a607-97bb-47ba-83ab-65e810f757c1 service placement] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:33.030535 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0910a607-97bb-47ba-83ab-65e810f757c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:33.036502 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0910a607-97bb-47ba-83ab-65e810f757c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:33.061368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcf94d-7de8-4fbd-b67f-a5fd891944b0 service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:33.062646 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910a607-97bb-47ba-83ab-65e810f757c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:33.066663 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:33.067798 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0910a607-97bb-47ba-83ab-65e810f757c1 service placement] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:33.068087 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:33.068546 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:33.068989 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] PATH_INFO: `/v3/users/cc70414d78304179ad171036ae225bfc` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:33.086372 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc70414d78304179ad171036ae225bfc)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:33.089581 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:33.101652 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1426/2867] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:33.179948 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc70414d78304179ad171036ae225bfc'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.180514 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc70414d78304179ad171036ae225bfc'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.182117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc70414d78304179ad171036ae225bfc'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.191024 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc70414d78304179ad171036ae225bfc'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.203025 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc70414d78304179ad171036ae225bfc'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.227138 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1442/2868] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:26:32 2022] DELETE /identity/v3/users/cc70414d78304179ad171036ae225bfc => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:26:33.343615 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8c1b8-a3ab-4a90-bbb8-6796ae34f1d4 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:33.344028 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8c1b8-a3ab-4a90-bbb8-6796ae34f1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:33.344431 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8c1b8-a3ab-4a90-bbb8-6796ae34f1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:33.362225 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-05f8c1b8-a3ab-4a90-bbb8-6796ae34f1d4 None None] MFA Rules not processed for user `c92576b377794d099fb0920e3b973293`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:33.363471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-05f8c1b8-a3ab-4a90-bbb8-6796ae34f1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:33.401868 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:33.403256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:33.428564 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1427/2869] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:33.479027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587f822-d851-4672-ad53-08417a3f63cb admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:33.483770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:33.484644 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:33.484936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:33.485220 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] PATH_INFO: `/v3/projects/f4a1a83049f34f45b51cae80f56bc65f` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:33.496222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a1a83049f34f45b51cae80f56bc65f)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:33.498163 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:33.510884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:33.551755 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a1a83049f34f45b51cae80f56bc65f', 'request_id': 'req-95c48788-8b9f-41b6-8316-2491d603fac9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.559535 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a1a83049f34f45b51cae80f56bc65f', 'request_id': 'req-95c48788-8b9f-41b6-8316-2491d603fac9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.566480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a1a83049f34f45b51cae80f56bc65f', 'request_id': 'req-95c48788-8b9f-41b6-8316-2491d603fac9'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.577294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] The token cache is being invalidate because project f4a1a83049f34f45b51cae80f56bc65f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:26:33.577584 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.578228 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.579857 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1443/2870] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:26:33 2022] DELETE /identity/v3/projects/f4a1a83049f34f45b51cae80f56bc65f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:26:33.583718 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8c1b8-a3ab-4a90-bbb8-6796ae34f1d4 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:33.595325 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8c1b8-a3ab-4a90-bbb8-6796ae34f1d4 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:33.650976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:33.651890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:33.652182 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:33.652459 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] PATH_INFO: `/v3/users/c9e6785d9ac2415e990f9d348f4d3b0b` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:33.666469 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e6785d9ac2415e990f9d348f4d3b0b)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:33.668825 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:33.691390 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:33.726918 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e6785d9ac2415e990f9d348f4d3b0b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.728463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e6785d9ac2415e990f9d348f4d3b0b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.729101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e6785d9ac2415e990f9d348f4d3b0b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.737803 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e6785d9ac2415e990f9d348f4d3b0b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.746017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-1ead9ebd-db5c-4fbb-b5bd-9550c291e46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e6785d9ac2415e990f9d348f4d3b0b'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:33.765039 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1428/2871] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:26:33 2022] DELETE /identity/v3/users/c9e6785d9ac2415e990f9d348f4d3b0b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:26:33.954190 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:33.956014 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:33.972257 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c48788-8b9f-41b6-8316-2491d603fac9 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:34.043579 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:34.045130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:34.045130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:34.045348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] PATH_INFO: `/v3/projects/79248ec3397046b7a6c4b36d9ef464e5` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:34.057893 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79248ec3397046b7a6c4b36d9ef464e5)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:34.060999 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:34.118580 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79248ec3397046b7a6c4b36d9ef464e5', 'request_id': 'req-c239e89c-0089-4154-a50a-3e2875fe7ff5'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:34.127239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79248ec3397046b7a6c4b36d9ef464e5', 'request_id': 'req-c239e89c-0089-4154-a50a-3e2875fe7ff5'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:34.134104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79248ec3397046b7a6c4b36d9ef464e5', 'request_id': 'req-c239e89c-0089-4154-a50a-3e2875fe7ff5'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:34.145390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] The token cache is being invalidate because project 79248ec3397046b7a6c4b36d9ef464e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:26:34.145698 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:34.146505 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:34.148281 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1444/2872] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:26:33 2022] DELETE /identity/v3/projects/79248ec3397046b7a6c4b36d9ef464e5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:26:43.761067 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:DELETE=8,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:44.142113 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=75,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:50.648274 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c540ef-7834-476d-b19c-213a176e3614 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:50.648757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c540ef-7834-476d-b19c-213a176e3614 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:50.648866 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c540ef-7834-476d-b19c-213a176e3614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:50.704976 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-51c540ef-7834-476d-b19c-213a176e3614 None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:50.706166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-51c540ef-7834-476d-b19c-213a176e3614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:50.770376 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1429/2873] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:26:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:50.815490 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:50.816920 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:50.906065 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c239e89c-0089-4154-a50a-3e2875fe7ff5 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:50.911031 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae14e4-2e58-4a0e-8b44-0dd9fa9c5e4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:50.911695 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae14e4-2e58-4a0e-8b44-0dd9fa9c5e4d service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:50.911981 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae14e4-2e58-4a0e-8b44-0dd9fa9c5e4d service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:50.912259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae14e4-2e58-4a0e-8b44-0dd9fa9c5e4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:50.920968 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae14e4-2e58-4a0e-8b44-0dd9fa9c5e4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:50.922583 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ae14e4-2e58-4a0e-8b44-0dd9fa9c5e4d service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:50.930805 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1445/2874] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:26:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:26:51.419508 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fcb2d-cfe9-486b-a428-cc93edb15d33 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:51.419811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fcb2d-cfe9-486b-a428-cc93edb15d33 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:51.420094 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fcb2d-cfe9-486b-a428-cc93edb15d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:51.437608 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-081fcb2d-cfe9-486b-a428-cc93edb15d33 None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:51.438811 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-081fcb2d-cfe9-486b-a428-cc93edb15d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:51.445885 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1430/2875] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jul 5 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:51.458160 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae14e4-2e58-4a0e-8b44-0dd9fa9c5e4d service swift] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:51.469647 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae14e4-2e58-4a0e-8b44-0dd9fa9c5e4d service swift] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:51.474640 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3c4ed-6509-4442-8c40-4d9774449151 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:51.475308 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3c4ed-6509-4442-8c40-4d9774449151 service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:51.475601 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3c4ed-6509-4442-8c40-4d9774449151 service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:51.475884 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3c4ed-6509-4442-8c40-4d9774449151 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:51.484110 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3c4ed-6509-4442-8c40-4d9774449151 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:51.485700 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3c4ed-6509-4442-8c40-4d9774449151 service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:51.494155 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1446/2876] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:26:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:26:51.676597 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925a358-e48e-4dcc-b44e-6b36158d4239 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:51.677068 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925a358-e48e-4dcc-b44e-6b36158d4239 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:51.677193 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b925a358-e48e-4dcc-b44e-6b36158d4239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:51.694720 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-b925a358-e48e-4dcc-b44e-6b36158d4239 None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:51.695939 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-b925a358-e48e-4dcc-b44e-6b36158d4239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:51.703589 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1431/2877] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 5 15:26:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:51.715099 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3c4ed-6509-4442-8c40-4d9774449151 service swift] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:51.725770 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3c4ed-6509-4442-8c40-4d9774449151 service swift] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:51.730758 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9905ff-1d91-4693-a628-94a0d7adcd8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:51.731432 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9905ff-1d91-4693-a628-94a0d7adcd8e service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:51.731722 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9905ff-1d91-4693-a628-94a0d7adcd8e service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:51.732002 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9905ff-1d91-4693-a628-94a0d7adcd8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:51.739934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9905ff-1d91-4693-a628-94a0d7adcd8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:51.741519 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9905ff-1d91-4693-a628-94a0d7adcd8e service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:51.749955 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1447/2878] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:26:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:26:52.057855 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e318b6-018b-47bc-a9e4-f46a0219333f None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:52.058151 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e318b6-018b-47bc-a9e4-f46a0219333f None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:52.058436 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e318b6-018b-47bc-a9e4-f46a0219333f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:52.076757 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-f4e318b6-018b-47bc-a9e4-f46a0219333f None None] MFA Rules not processed for user `b3ab5ed3d9e341f7853302ac7826879b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:52.077964 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f4e318b6-018b-47bc-a9e4-f46a0219333f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:52.085188 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1432/2879] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jul 5 15:26:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:52.097227 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9905ff-1d91-4693-a628-94a0d7adcd8e service swift] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:52.108252 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9905ff-1d91-4693-a628-94a0d7adcd8e service swift] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:52.113247 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7205c3-21d7-45b0-94ea-5bfc00dc253a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4c73aa1f244d4251b8539c2806f38a42', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:52.113883 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7205c3-21d7-45b0-94ea-5bfc00dc253a service swift] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:52.114172 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7205c3-21d7-45b0-94ea-5bfc00dc253a service swift] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:52.114452 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7205c3-21d7-45b0-94ea-5bfc00dc253a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:52.123017 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7205c3-21d7-45b0-94ea-5bfc00dc253a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:52.124632 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7205c3-21d7-45b0-94ea-5bfc00dc253a service swift] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:52.132603 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1448/2880] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 5 15:26:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 05 15:26:54.148252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e318b6-018b-47bc-a9e4-f46a0219333f None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:54.149791 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f4e318b6-018b-47bc-a9e4-f46a0219333f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:54.213914 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e318b6-018b-47bc-a9e4-f46a0219333f None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:54.218367 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:54.219556 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:54.219836 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:54.220112 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] PATH_INFO: `/v3/users/c92576b377794d099fb0920e3b973293` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:54.221297 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92576b377794d099fb0920e3b973293)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:54.223253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:54.267495 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:54.269047 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92576b377794d099fb0920e3b973293'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:54.269773 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92576b377794d099fb0920e3b973293'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:54.270676 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92576b377794d099fb0920e3b973293'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:54.278447 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92576b377794d099fb0920e3b973293'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:54.285363 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92576b377794d099fb0920e3b973293'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:54.302100 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1433/2881] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:26:54 2022] DELETE /identity/v3/users/c92576b377794d099fb0920e3b973293 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:26:54.306788 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be52e0-3081-455c-a524-1b0375b9207c None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:54.307294 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be52e0-3081-455c-a524-1b0375b9207c None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:54.307995 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be52e0-3081-455c-a524-1b0375b9207c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:54.325645 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-26be52e0-3081-455c-a524-1b0375b9207c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:54.326832 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-26be52e0-3081-455c-a524-1b0375b9207c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:54.382310 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1449/2882] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:54.392413 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:54.393847 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:54.451366 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afc86f5-ff34-4c95-b342-7f23a909268a admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:54.454930 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092941cf-ba70-47ec-9473-7b5381785a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:54.455603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092941cf-ba70-47ec-9473-7b5381785a20 service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:54.455888 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092941cf-ba70-47ec-9473-7b5381785a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:54.456166 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092941cf-ba70-47ec-9473-7b5381785a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:54.465584 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092941cf-ba70-47ec-9473-7b5381785a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:54.467009 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092941cf-ba70-47ec-9473-7b5381785a20 service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:54.476548 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1434/2883] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:54.642986 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be52e0-3081-455c-a524-1b0375b9207c None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:54.644524 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-26be52e0-3081-455c-a524-1b0375b9207c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:54.656933 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be52e0-3081-455c-a524-1b0375b9207c None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:54.662147 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:54.663341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:54.663341 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:54.663606 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] PATH_INFO: `/v3/projects/c7984f018652497b8844155091acd2c3` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:54.664940 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7984f018652497b8844155091acd2c3)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:54.666834 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:54.698941 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:54.721477 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7984f018652497b8844155091acd2c3', 'request_id': 'req-1212b402-9211-4f9f-8db9-9d341d1ae031'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:54.728459 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7984f018652497b8844155091acd2c3', 'request_id': 'req-1212b402-9211-4f9f-8db9-9d341d1ae031'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:54.734633 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7984f018652497b8844155091acd2c3', 'request_id': 'req-1212b402-9211-4f9f-8db9-9d341d1ae031'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:54.745398 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] The token cache is being invalidate because project c7984f018652497b8844155091acd2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80351) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:26:54.745693 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:54.746347 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-1212b402-9211-4f9f-8db9-9d341d1ae031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:26:54.747918 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1450/2884] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:26:54 2022] DELETE /identity/v3/projects/c7984f018652497b8844155091acd2c3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:26:54.752988 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d80b07-fc08-47d6-b121-d62b0805b35b None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:54.753308 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d80b07-fc08-47d6-b121-d62b0805b35b None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:54.753590 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d80b07-fc08-47d6-b121-d62b0805b35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:54.771733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-02d80b07-fc08-47d6-b121-d62b0805b35b None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:54.772853 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-02d80b07-fc08-47d6-b121-d62b0805b35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:54.829063 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1435/2885] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:54.837497 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf38bc4-bf74-44e4-bb67-08321f337a4e None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:54.837799 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf38bc4-bf74-44e4-bb67-08321f337a4e None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:54.838089 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf38bc4-bf74-44e4-bb67-08321f337a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:54.856685 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-ddf38bc4-bf74-44e4-bb67-08321f337a4e None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:54.857902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-ddf38bc4-bf74-44e4-bb67-08321f337a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:54.865835 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1451/2886] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:54.870463 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d80b07-fc08-47d6-b121-d62b0805b35b None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:54.879871 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d80b07-fc08-47d6-b121-d62b0805b35b None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:54.884427 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24647195-d1e8-4e92-88f9-168cdcfb1415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:54.885071 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24647195-d1e8-4e92-88f9-168cdcfb1415 admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:54.885356 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24647195-d1e8-4e92-88f9-168cdcfb1415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:54.885632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24647195-d1e8-4e92-88f9-168cdcfb1415 admin admin] PATH_INFO: `/v3/domains` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:54.886354 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24647195-d1e8-4e92-88f9-168cdcfb1415 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80352) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 05 15:26:54.886655 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24647195-d1e8-4e92-88f9-168cdcfb1415 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:54.888259 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24647195-d1e8-4e92-88f9-168cdcfb1415 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:54.900700 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1436/2887] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 5 15:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 05 15:26:54.903558 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf38bc4-bf74-44e4-bb67-08321f337a4e None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:54.912735 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf38bc4-bf74-44e4-bb67-08321f337a4e None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:54.917897 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a69b94-b0a8-43f2-b7fd-a1ea6e13a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:54.918542 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a69b94-b0a8-43f2-b7fd-a1ea6e13a543 admin admin] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:54.918860 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a69b94-b0a8-43f2-b7fd-a1ea6e13a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:54.919183 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a69b94-b0a8-43f2-b7fd-a1ea6e13a543 admin admin] PATH_INFO: `/v3/projects` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:54.919876 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a69b94-b0a8-43f2-b7fd-a1ea6e13a543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:54.921787 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a69b94-b0a8-43f2-b7fd-a1ea6e13a543 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:54.946789 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1452/2888] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 5 15:26:54 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:26:54.955252 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24647195-d1e8-4e92-88f9-168cdcfb1415 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:54.965767 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24647195-d1e8-4e92-88f9-168cdcfb1415 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:55.016859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:55.026894 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec26af8-ca02-4923-85a0-dcc338f5e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:55.027532 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec26af8-ca02-4923-85a0-dcc338f5e9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:55.027818 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec26af8-ca02-4923-85a0-dcc338f5e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:55.028101 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec26af8-ca02-4923-85a0-dcc338f5e9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:55.028802 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec26af8-ca02-4923-85a0-dcc338f5e9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:55.030769 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec26af8-ca02-4923-85a0-dcc338f5e9a2 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:55.049723 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1437/2889] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 5 15:26:54 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 05 15:26:55.053767 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a69b94-b0a8-43f2-b7fd-a1ea6e13a543 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:55.064527 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a69b94-b0a8-43f2-b7fd-a1ea6e13a543 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:55.069242 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6ace7-375b-4b9b-8900-bd6d53a0ce40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:55.069901 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6ace7-375b-4b9b-8900-bd6d53a0ce40 admin admin] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:55.070186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6ace7-375b-4b9b-8900-bd6d53a0ce40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:55.070480 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6ace7-375b-4b9b-8900-bd6d53a0ce40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:55.071330 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6ace7-375b-4b9b-8900-bd6d53a0ce40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:55.073105 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6ace7-375b-4b9b-8900-bd6d53a0ce40 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:55.084680 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1453/2890] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 05 15:26:55.088603 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec26af8-ca02-4923-85a0-dcc338f5e9a2 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:55.098959 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec26af8-ca02-4923-85a0-dcc338f5e9a2 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:55.104081 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a5f179-218b-4e16-b1a6-859634d3282e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:55.104733 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5f179-218b-4e16-b1a6-859634d3282e admin admin] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:55.105019 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5f179-218b-4e16-b1a6-859634d3282e admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:55.105300 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a5f179-218b-4e16-b1a6-859634d3282e admin admin] PATH_INFO: `/v3/roles` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:55.106053 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5f179-218b-4e16-b1a6-859634d3282e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:55.107679 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a5f179-218b-4e16-b1a6-859634d3282e admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:55.117411 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1438/2891] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 5 15:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 05 15:26:55.121838 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6ace7-375b-4b9b-8900-bd6d53a0ce40 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:55.131751 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6ace7-375b-4b9b-8900-bd6d53a0ce40 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:55.136325 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a21a66-2c0d-4215-b368-e173bf4bd03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:55.137198 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21a66-2c0d-4215-b368-e173bf4bd03b admin admin] REQUEST_METHOD: `PUT` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:55.137488 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21a66-2c0d-4215-b368-e173bf4bd03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:55.137771 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21a66-2c0d-4215-b368-e173bf4bd03b admin admin] PATH_INFO: `/v3/projects/71bc8a4cb44a4ef7b29cde5b4ac53c8f/users/5ec5c67ae1e94df58702f086eae711cb/roles/6e32448ba01f41bcafb4cb0a839691fe` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:55.152070 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a21a66-2c0d-4215-b368-e173bf4bd03b admin admin] RBAC: Authorizing `identity:create_grant(project_id=71bc8a4cb44a4ef7b29cde5b4ac53c8f, user_id=5ec5c67ae1e94df58702f086eae711cb, role_id=6e32448ba01f41bcafb4cb0a839691fe)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:55.154756 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a21a66-2c0d-4215-b368-e173bf4bd03b admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:55.167508 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1454/2892] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 5 15:26:55 2022] PUT /identity/v3/projects/71bc8a4cb44a4ef7b29cde5b4ac53c8f/users/5ec5c67ae1e94df58702f086eae711cb/roles/6e32448ba01f41bcafb4cb0a839691fe => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:26:55.172415 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298d0ac-8bdb-43f5-b867-2022396dc24c None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:55.172902 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298d0ac-8bdb-43f5-b867-2022396dc24c None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:55.173003 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d298d0ac-8bdb-43f5-b867-2022396dc24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:55.191999 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-d298d0ac-8bdb-43f5-b867-2022396dc24c None None] MFA Rules not processed for user `7b7d0fd6414b417ab8bf3499e0f4961b`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:55.193197 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d298d0ac-8bdb-43f5-b867-2022396dc24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:55.251662 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1439/2893] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:55.261271 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a21a66-2c0d-4215-b368-e173bf4bd03b admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:55.262965 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c6a21a66-2c0d-4215-b368-e173bf4bd03b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:55.321511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a21a66-2c0d-4215-b368-e173bf4bd03b admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:55.325161 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69008f3-2142-4810-b88c-4045ca64cc76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:55.325812 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69008f3-2142-4810-b88c-4045ca64cc76 service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:55.326104 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69008f3-2142-4810-b88c-4045ca64cc76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:55.326390 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69008f3-2142-4810-b88c-4045ca64cc76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:55.335348 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69008f3-2142-4810-b88c-4045ca64cc76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:55.336798 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69008f3-2142-4810-b88c-4045ca64cc76 service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:55.346874 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1455/2894] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:55.948216 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf34af-878d-49e2-8096-31918df87abc None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:55.948784 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf34af-878d-49e2-8096-31918df87abc None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:55.949270 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf34af-878d-49e2-8096-31918df87abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:55.978786 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-49bf34af-878d-49e2-8096-31918df87abc None None] MFA Rules not processed for user `5ec5c67ae1e94df58702f086eae711cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:55.980099 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-49bf34af-878d-49e2-8096-31918df87abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:56.048904 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1440/2895] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:26:56.060368 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69008f3-2142-4810-b88c-4045ca64cc76 service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:26:56.061801 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-f69008f3-2142-4810-b88c-4045ca64cc76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:56.120869 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69008f3-2142-4810-b88c-4045ca64cc76 service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:26:56.124417 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d8586-885c-4aef-bf8a-461e1f81947d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ad7cee9ffed415b9a30176b1888e562', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:26:56.125078 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d8586-885c-4aef-bf8a-461e1f81947d service cinder] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:56.125367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d8586-885c-4aef-bf8a-461e1f81947d service cinder] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:56.125652 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d8586-885c-4aef-bf8a-461e1f81947d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:56.133011 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:26:56.135724 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d8586-885c-4aef-bf8a-461e1f81947d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:26:56.138132 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d8586-885c-4aef-bf8a-461e1f81947d service cinder] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:26:56.148578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1456/2896] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:26:57.484809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883bbe6-5c17-4480-91e0-86f35cc3e382 None None] REQUEST_METHOD: `POST` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:26:57.485298 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883bbe6-5c17-4480-91e0-86f35cc3e382 None None] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:26:57.485417 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883bbe6-5c17-4480-91e0-86f35cc3e382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:26:57.502969 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.auth.core [None req-e883bbe6-5c17-4480-91e0-86f35cc3e382 None None] MFA Rules not processed for user `5ec5c67ae1e94df58702f086eae711cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80352) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:26:57.504199 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e883bbe6-5c17-4480-91e0-86f35cc3e382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:26:57.511641 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1441/2897] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:26:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 05 15:27:06.144929 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=2 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:27:07.497886 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:27:29.026744 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d8586-885c-4aef-bf8a-461e1f81947d service cinder] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:27:29.028196 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-c69d8586-885c-4aef-bf8a-461e1f81947d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:27:29.096574 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d8586-885c-4aef-bf8a-461e1f81947d service cinder] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:27:29.101578 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58dda6-7fe6-420e-8405-e75fb7dd8ae2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:27:29.102222 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58dda6-7fe6-420e-8405-e75fb7dd8ae2 service placement] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:27:29.102511 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58dda6-7fe6-420e-8405-e75fb7dd8ae2 service placement] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:27:29.102793 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58dda6-7fe6-420e-8405-e75fb7dd8ae2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:27:29.104562 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-6e58dda6-7fe6-420e-8405-e75fb7dd8ae2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:27:29.116186 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58dda6-7fe6-420e-8405-e75fb7dd8ae2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:27:29.117852 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58dda6-7fe6-420e-8405-e75fb7dd8ae2 service placement] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:27:29.128440 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1457/2898] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:27:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:27:39.123572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=32 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:28:12.047747 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883bbe6-5c17-4480-91e0-86f35cc3e382 None None] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:28:12.049201 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-e883bbe6-5c17-4480-91e0-86f35cc3e382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:28:12.061077 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883bbe6-5c17-4480-91e0-86f35cc3e382 None None] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:28:12.064550 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f24f7e-b80c-4d3e-b906-0c3f96b9497b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:28:12.065186 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24f7e-b80c-4d3e-b906-0c3f96b9497b service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:28:12.065471 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24f7e-b80c-4d3e-b906-0c3f96b9497b service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:28:12.065752 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f24f7e-b80c-4d3e-b906-0c3f96b9497b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:28:12.067472 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-d8f24f7e-b80c-4d3e-b906-0c3f96b9497b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:28:12.127399 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f24f7e-b80c-4d3e-b906-0c3f96b9497b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:28:12.128846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f24f7e-b80c-4d3e-b906-0c3f96b9497b service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:28:12.137913 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1442/2899] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:28:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:28:13.635130 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb008f-43a9-4743-a93c-2eddd49617d4 None None] REQUEST_METHOD: `POST` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:28:13.635667 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb008f-43a9-4743-a93c-2eddd49617d4 None None] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:28:13.635969 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb008f-43a9-4743-a93c-2eddd49617d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:28:13.653611 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.auth.core [None req-51bb008f-43a9-4743-a93c-2eddd49617d4 None None] MFA Rules not processed for user `5ec5c67ae1e94df58702f086eae711cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80351) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 05 15:28:13.654822 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-51bb008f-43a9-4743-a93c-2eddd49617d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:28:13.662458 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1458/2900] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 5 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 05 15:28:13.895710 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f24f7e-b80c-4d3e-b906-0c3f96b9497b service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:28:13.904272 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f24f7e-b80c-4d3e-b906-0c3f96b9497b service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:28:13.907848 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0c884-30d4-4c03-8805-e647d5b2abda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:28:13.908482 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0c884-30d4-4c03-8805-e647d5b2abda service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:28:13.908763 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0c884-30d4-4c03-8805-e647d5b2abda service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:28:13.909038 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0c884-30d4-4c03-8805-e647d5b2abda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:28:13.910790 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-14e0c884-30d4-4c03-8805-e647d5b2abda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:28:13.920632 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0c884-30d4-4c03-8805-e647d5b2abda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:28:13.922049 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e0c884-30d4-4c03-8805-e647d5b2abda service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:28:13.930530 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1443/2901] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:28:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:28:16.836304 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb008f-43a9-4743-a93c-2eddd49617d4 None None] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:28:16.846320 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb008f-43a9-4743-a93c-2eddd49617d4 None None] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:28:16.849992 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86720b5-4c3c-4f01-901d-1e31adb835cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:28:16.850690 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86720b5-4c3c-4f01-901d-1e31adb835cf service neutron] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:28:16.850976 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86720b5-4c3c-4f01-901d-1e31adb835cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:28:16.851293 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86720b5-4c3c-4f01-901d-1e31adb835cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:28:16.853845 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a86720b5-4c3c-4f01-901d-1e31adb835cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:28:16.935610 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86720b5-4c3c-4f01-901d-1e31adb835cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:28:16.937117 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86720b5-4c3c-4f01-901d-1e31adb835cf service neutron] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:28:16.948912 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1459/2902] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:28:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:28:18.258119 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0c884-30d4-4c03-8805-e647d5b2abda service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:28:18.266890 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0c884-30d4-4c03-8805-e647d5b2abda service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:28:18.270477 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9ee0f-307e-48c8-a6ce-180daafd908b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd7fa93466c54632ad95834297410e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:28:18.271777 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9ee0f-307e-48c8-a6ce-180daafd908b service neutron] REQUEST_METHOD: `GET` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:28:18.272066 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9ee0f-307e-48c8-a6ce-180daafd908b service neutron] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:28:18.272340 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9ee0f-307e-48c8-a6ce-180daafd908b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:28:18.274114 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-f1a9ee0f-307e-48c8-a6ce-180daafd908b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:28:18.285756 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9ee0f-307e-48c8-a6ce-180daafd908b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:28:18.287165 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9ee0f-307e-48c8-a6ce-180daafd908b service neutron] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:28:18.296588 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1444/2903] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:28:25.355086 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86720b5-4c3c-4f01-901d-1e31adb835cf service neutron] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:28:25.356708 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-a86720b5-4c3c-4f01-901d-1e31adb835cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:28:25.373453 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86720b5-4c3c-4f01-901d-1e31adb835cf service neutron] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:28:25.378214 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:28:25.379234 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:28:25.379587 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:28:25.379928 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] PATH_INFO: `/v3/users/2891dbd54dfb47e4a98d47d2d78b7edf` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:28:25.381203 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2891dbd54dfb47e4a98d47d2d78b7edf)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:28:25.383239 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:28:25.435818 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2891dbd54dfb47e4a98d47d2d78b7edf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.436517 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2891dbd54dfb47e4a98d47d2d78b7edf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.437174 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2891dbd54dfb47e4a98d47d2d78b7edf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.444290 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2891dbd54dfb47e4a98d47d2d78b7edf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.451522 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2891dbd54dfb47e4a98d47d2d78b7edf'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.470848 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1460/2904] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:28:25 2022] DELETE /identity/v3/users/2891dbd54dfb47e4a98d47d2d78b7edf => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:28:25.641744 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9ee0f-307e-48c8-a6ce-180daafd908b service neutron] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:28:25.652501 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9ee0f-307e-48c8-a6ce-180daafd908b service neutron] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:28:25.715291 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:28:25.716154 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:28:25.716442 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:28:25.716724 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] PATH_INFO: `/v3/projects/32d1debb0ad2409f9b74b0499489f4fc` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:28:25.717898 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d1debb0ad2409f9b74b0499489f4fc)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:28:25.720135 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:28:25.764041 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d1debb0ad2409f9b74b0499489f4fc', 'request_id': 'req-630c4171-514e-4985-918c-fd3a5d0a8147'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.770672 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d1debb0ad2409f9b74b0499489f4fc', 'request_id': 'req-630c4171-514e-4985-918c-fd3a5d0a8147'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.777224 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d1debb0ad2409f9b74b0499489f4fc', 'request_id': 'req-630c4171-514e-4985-918c-fd3a5d0a8147'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.787196 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] The token cache is being invalidate because project 32d1debb0ad2409f9b74b0499489f4fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:28:25.787489 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.788111 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.789613 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1445/2905] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:28:25 2022] DELETE /identity/v3/projects/32d1debb0ad2409f9b74b0499489f4fc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:28:25.793669 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:28:25.795069 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:28:25.807925 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedaa36-d67b-4f48-9b09-10bd173f78e1 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:28:25.854833 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:28:25.865589 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:28:25.866461 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:28:25.866745 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:28:25.867059 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] PATH_INFO: `/v3/users/52315700f2a8435abe22ed8a995d6275` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:28:25.868243 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52315700f2a8435abe22ed8a995d6275)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:28:25.870159 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:28:25.914460 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52315700f2a8435abe22ed8a995d6275'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.915176 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52315700f2a8435abe22ed8a995d6275'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.915777 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52315700f2a8435abe22ed8a995d6275'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.922567 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52315700f2a8435abe22ed8a995d6275'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.929047 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52315700f2a8435abe22ed8a995d6275'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:25.946408 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1461/2906] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:28:25 2022] DELETE /identity/v3/users/52315700f2a8435abe22ed8a995d6275 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:28:26.124309 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:28:26.125809 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:28:26.135721 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:28:26.143911 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c4171-514e-4985-918c-fd3a5d0a8147 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:28:26.210528 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:28:26.211445 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:28:26.211726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:28:26.212001 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] PATH_INFO: `/v3/projects/3433b29956f54c6c944f73d80642b663` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:28:26.213183 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3433b29956f54c6c944f73d80642b663)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:28:26.215104 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:28:26.261901 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3433b29956f54c6c944f73d80642b663', 'request_id': 'req-5388177b-818c-4753-81bf-46d8b27b1301'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:26.269488 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3433b29956f54c6c944f73d80642b663', 'request_id': 'req-5388177b-818c-4753-81bf-46d8b27b1301'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:26.276726 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3433b29956f54c6c944f73d80642b663', 'request_id': 'req-5388177b-818c-4753-81bf-46d8b27b1301'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:26.288253 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] The token cache is being invalidate because project 3433b29956f54c6c944f73d80642b663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:28:26.288665 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:26.289377 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:28:26.291251 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1446/2907] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:28:26 2022] DELETE /identity/v3/projects/3433b29956f54c6c944f73d80642b663 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:28:35.951703 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:28:36.285231 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:29:17.466321 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:29:17.467902 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:29:17.480907 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed520cb-eb3d-40b3-9476-428e082123c7 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:29:17.538128 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:29:17.538937 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:29:17.539256 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:29:17.539539 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] PATH_INFO: `/v3/users/5ec5c67ae1e94df58702f086eae711cb` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:29:17.540717 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ec5c67ae1e94df58702f086eae711cb)` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:29:17.543166 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:29:17.587306 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ec5c67ae1e94df58702f086eae711cb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:29:17.587950 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ec5c67ae1e94df58702f086eae711cb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:29:17.588572 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ec5c67ae1e94df58702f086eae711cb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:29:17.596219 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ec5c67ae1e94df58702f086eae711cb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:29:17.603600 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.notifications [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ec5c67ae1e94df58702f086eae711cb'} {{(pid=80351) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:29:17.622133 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1462/2908] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 5 15:29:17 2022] DELETE /identity/v3/users/5ec5c67ae1e94df58702f086eae711cb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:29:17.791017 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] Authenticating user token {{(pid=80352) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:29:17.792846 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.fernet_utils [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80352) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:29:17.805949 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388177b-818c-4753-81bf-46d8b27b1301 admin admin] Validating token access rules against request {{(pid=80352) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:29:17.862970 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7d0fd6414b417ab8bf3499e0f4961b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77588a8e554a4d32b7cfdcc698266c19', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80352) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:29:17.863859 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:29:17.864143 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:29:17.864418 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] PATH_INFO: `/v3/projects/71bc8a4cb44a4ef7b29cde5b4ac53c8f` {{(pid=80352) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:29:17.865592 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71bc8a4cb44a4ef7b29cde5b4ac53c8f)` {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:29:17.867553 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] RBAC: Authorization granted {{(pid=80352) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:29:17.911643 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71bc8a4cb44a4ef7b29cde5b4ac53c8f', 'request_id': 'req-76c5a183-d837-4f8d-8423-f66edf4e1bb4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:29:17.919015 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71bc8a4cb44a4ef7b29cde5b4ac53c8f', 'request_id': 'req-76c5a183-d837-4f8d-8423-f66edf4e1bb4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:29:17.925118 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71bc8a4cb44a4ef7b29cde5b4ac53c8f', 'request_id': 'req-76c5a183-d837-4f8d-8423-f66edf4e1bb4'} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:29:17.938938 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] The token cache is being invalidate because project 71bc8a4cb44a4ef7b29cde5b4ac53c8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80352) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 05 15:29:17.939256 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:29:17.939903 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG keystone.notifications [None req-76c5a183-d837-4f8d-8423-f66edf4e1bb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80352) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 05 15:29:17.941419 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: [pid: 80352|app: 0|req: 1447/2909] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 5 15:29:17 2022] DELETE /identity/v3/projects/71bc8a4cb44a4ef7b29cde5b4ac53c8f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 05 15:29:27.618027 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:29:27.932942 dv-cinder-841125-10-iscsi devstack@keystone.service[80352]: DEBUG dbcounter [-] [80352] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80352) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 05 15:32:31.978259 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] Authenticating user token {{(pid=80351) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 05 15:32:31.979836 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:32:32.048364 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7469b36a-20a1-4137-9740-c11941e33635 admin admin] Validating token access rules against request {{(pid=80351) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 05 15:32:32.106292 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1441d-336e-4df6-84e9-08601d3db242 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c68f1297771e4c03a897dd1f0ac21cea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d232251a8b245c2a547b7bc59bffb43', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80351) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 05 15:32:32.106936 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1441d-336e-4df6-84e9-08601d3db242 service placement] REQUEST_METHOD: `GET` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 05 15:32:32.107323 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1441d-336e-4df6-84e9-08601d3db242 service placement] SCRIPT_NAME: `/identity` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 05 15:32:32.107605 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1441d-336e-4df6-84e9-08601d3db242 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80351) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 05 15:32:32.109367 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.fernet_utils [None req-beb1441d-336e-4df6-84e9-08601d3db242 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80351) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 05 15:32:32.120345 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1441d-336e-4df6-84e9-08601d3db242 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 05 15:32:32.121934 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1441d-336e-4df6-84e9-08601d3db242 service placement] RBAC: Authorization granted {{(pid=80351) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 05 15:32:32.131673 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: [pid: 80351|app: 0|req: 1463/2910] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 5 15:32:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 05 15:32:42.127483 dv-cinder-841125-10-iscsi devstack@keystone.service[80351]: DEBUG dbcounter [-] [80351] Writing DB stats keystone:SELECT=58 {{(pid=80351) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}